diff options
Diffstat (limited to 'controller')
-rw-r--r-- | controller/README.md | 257 | ||||
-rw-r--r-- | controller/SqliteNetworkController.cpp | 303 | ||||
-rw-r--r-- | controller/SqliteNetworkController.hpp | 1 |
3 files changed, 441 insertions, 120 deletions
diff --git a/controller/README.md b/controller/README.md index 6989e98d..8b789a3e 100644 --- a/controller/README.md +++ b/controller/README.md @@ -1,32 +1,261 @@ -Network Controller Implementation +Network Controller Microservice ====== -This folder contains code implementing the node/NetworkController.hpp interface to allow ZeroTier nodes to create and manage virtual networks. +ZeroTier's 16-digit network IDs are really just a concatenation of the 10-digit ZeroTier address of a network controller followed by a 6-digit (24-bit) network number on that controller. Fans of software defined networking will recognize this as a variation of the familiar [separation of data plane and control plane](http://sdntutorials.com/difference-between-control-plane-and-data-plane/) SDN design pattern. + +This code implements the *node/NetworkController.hpp* interface and provides a SQLite3-backed network controller microservice. Including it in the build allows ZeroTier One to act as a controller and create/manage networks. + +This is the same code we use to run [my.zerotier.com](https://my.zerotier.com/), which is a web UI and API that runs in front of a pool of controllers. ### Building -By default this code is not built or included in the client. To build on Linux, BSD, or Mac add ZT_\ENABLE_\NETWORK_\CONTROLLER=1 to the make command line. You'll need the development headers for Sqlite3 installed. They ship as part of OSX and Xcode. On Linux or BSD you'll probably need to install a package. +On Linux, Mac, or BSD you can create a controller-enabled build with: + + make ZT_ENABLE_NETWORK_CONTROLLER=1 + +You will need the development headers and libraries for SQLite3 installed. ### Running -When started, a controller-enabled build of ZeroTier One will automatically create and initialize a *controller.db* in its home folder. This is where all the controller's data and persistent state lives. +After building and installing (`make install`) a controller-enabled build of ZeroTier One, start it and try: + + sudo zerotier-cli /controller + +You should see something like: + + { + "controller": true, + "apiVersion": 2, + "clock": 1468002975497, + "instanceId": "8ab354604debe1da27ee627c9ef94a48" + } + +When started, a controller-enabled build of ZeroTier One will automatically create and initialize a `controller.db` file in its home folder. This is where all the controller's data and persistent state lives. If you're upgrading an old controller it will upgrade its database schema automatically on first launch. Make a backup of the old controller's database first since you can't go backward. + +Controllers periodically make backups of their database as `controller.db.backup`. This is done so that this file can be more easily copied/rsync'ed to other systems without worrying about corruption. SQLite3 supports multiple processes accessing the same database file, so `sqlite3 /path/to/controller.db .dump` also works but can be slow on a busy controller. + +Controllers can in theory host up to 2^24 networks and serve many millions of devices (or more), but we recommend running multiple controllers for a lot of networks to spread load and be more fault tolerant. + +### Dockerizing Controllers + +ZeroTier network controllers can easily be run in Docker or other container systems. Since containers do not need to actually join networks, extra privilege options like "--device=/dev/net/tun --privileged" are not needed. You'll just need to map the local JSON API port of the running controller and allow it to access the Internet (over UDP/9993 at a minimum) so things can reach and query it. + +### Implementing High Availability Fail-Over + +ZeroTier network controllers are not single points of failure for networks-- in the sense that if a controller goes down *existing* members of a network can continue to communicate. But new members (or those that have been offline for a while) can't join, existing members can't be de-authorized, and other changes to the network's configuration can't be made. This means that short "glitches" in controller availability are not a major problem but long periods of unavailability can be. + +Because controllers are just regular ZeroTier nodes and controller queries are in-band, controllers can trivially be moved without worrying about changes to underlying physical IPs. This makes high-availability fail-over very easy to implement. + +Just set up two cloud hosts, preferably in different data centers (e.g. two different AWS regions or Digital Ocean SF and NYC). Now set up the hot spare controller to constantly mirror `controller.db.backup` from its active sibling. + +If the active controller goes down, rename `controller.db.backup` to `controller.db` on the hot spare and start the ZeroTier One service there. The spare will take over and has now become the active controller. If the original active node comes back, it should take on the role of spare and should not start its service. Instead it should start mirroring the active controller's backup and wait until it is needed. + +The details of actually implementing this kind of HA fail-over on Linux or other OSes are beyond the scope of these docs and there are many ways to do it. Docker orchestration tools like Kubernetes can also be used to accomplish this if you've dockerized your controller. + +### Network Controller API + +The controller API is hosted via the same JSON API endpoint that ZeroTier One uses for local control (usually at 127.0.0.1 port 9993). All controller options are routed under the `/controller` base path. + +The controller microservice does not implement any fine-grained access control (authentication is via authtoken.secret just like the regular JSON API) or other complex mangement features. It just takes network and network member configurations and reponds to controller queries. We have an enterprise product called [ZeroTier Central](https://my.zerotier.com/) that we host as a service (and that companies can license to self-host) that does this. + +All working network IDs on a controller must begin with the controller's ZeroTier address. The API will *allow* "foreign" networks to be added but the controller will have no way of doing anything with them. + +Also note that the API is *very* sensitive about types. Integers must be integers and strings strings, etc. Incorrectly typed and unrecognized fields are just ignored. + +#### `/controller` + + * Purpose: Check for controller function and return controller status + * Methods: GET + * Returns: { object } + +| Field | Type | Description | Writable | +| ------------------ | ----------- | ------------------------------------------------- | -------- | +| controller | boolean | Always 'true' | no | +| apiVersion | integer | Controller API version, currently 2 | no | +| clock | integer | Current clock on controller, ms since epoch | no | +| instanceId | string | A random ID generated on first controller DB init | no | + +The instance ID can be used to check whether a controller's database has been reset or otherwise switched. + +#### `/controller/network` + + * Purpose: List all networks hosted by this controller + * Methods: GET + * Returns: [ string, ... ] + +This returns an array of 16-digit hexadecimal network IDs. + +#### `/controller/network/<network ID>` + + * Purpose: Create, configure, and delete hosted networks + * Methods: GET, POST, DELETE + * Returns: { object } + +By making queries to this path you can create, configure, and delete networks. DELETE is final, so don't do it unless you really mean it. + +When POSTing new networks take care that their IDs are not in use, otherwise you may overwrite an existing one. To create a new network with a random unused ID, POST to `/controller/network/##########______`. The #'s are the controller's 10-digit ZeroTier address and they're followed by six underscores. Check the `nwid` field of the returned JSON object for your network's newly allocated ID. Subsequent POSTs to this network must refer to its actual path. + +| Field | Type | Description | Writable | +| --------------------- | ------------- | ------------------------------------------------- | -------- | +| nwid | string | 16-digit network ID | no | +| controllerInstanceId | string | Controller database instance ID | no | +| clock | integer | Current clock, ms since epoch | no | +| name | string | A short name for this network | YES | +| private | boolean | Is access control enabled? | YES | +| enableBroadcast | boolean | Ethernet ff:ff:ff:ff:ff:ff allowed? | YES | +| allowPassiveBridging | boolean | Allow any member to bridge (very experimental) | YES | +| v4AssignMode | string | If 'zt', auto-assign IPv4 from pool(s) | YES | +| v6AssignMode | string | IPv6 address auto-assign modes; see below | YES | +| multicastLimit | integer | Maximum recipients for a multicast packet | YES | +| creationTime | integer | Time network was first created | no | +| revision | integer | Network config revision counter | no | +| memberRevisionCounter | integer | Network member revision counter | no | +| authorizedMemberCount | integer | Number of authorized members (for private nets) | no | +| relays | array[object] | Alternative relays; see below | YES | +| routes | array[object] | Managed IPv4 and IPv6 routes; see below | YES | +| ipAssignmentPools | array[object] | IP auto-assign ranges; see below | YES | +| rules | array[object] | Traffic rules; see below | YES | + +(The `ipLocalRoutes` field appeared in older versions but is no longer present. Routes will now show up in `routes`.) + +Two important things to know about networks: + + - Networks without rules won't carry any traffic. See below for an example with rules to permit IPv4 and IPv6. + - Managed IP address assignments and IP assignment pools that do not fall within a route configured in `routes` are ignored and won't be used or sent to members. + - The default for `private` is `true` and this is probably what you want. Turning `private` off means *anyone* can join your network with only its 16-digit network ID. It's also impossible to de-authorize a member as these networks don't issue or enforce certificates. Such "party line" networks are used for decentralized app backplanes, gaming, and testing but are not common in ordinary use. + +**IPv6 Auto-Assign Modes:** + +This field is (for legacy reasons) a comma-delimited list of strings. These can be `rfc4193`, `6plane`, and `zt`. RFC4193 and 6PLANE are special addressing modes that deterministically assign IPv6 addresses based on the network ID and the ZeroTier address of each member. The `zt` mode enables IPv6 auto-assignment from arbitrary IPv6 IP ranges configured in `ipAssignmentPools`. + +**Relay object format:** + +Relay objects define network-specific preferred relay nodes. Traffic to peers on this network will preferentially use these relays if they are available, and otherwise will fall back to the global rootserver infrastructure. + +| Field | Type | Description | Writable | +| --------------------- | ------------- | ------------------------------------------------- | -------- | +| address | string | 10-digit ZeroTier address of relay | YES | +| phyAddress | string | Optional IP/port suggestion for finding relay | YES | + +**IP assignment pool object format:** + +| Field | Type | Description | Writable | +| --------------------- | ------------- | ------------------------------------------------- | -------- | +| ipRangeStart | string | Starting IP address in range | YES | +| ipRangeEnd | string | Ending IP address in range (inclusive) | YES | + +Pools are only used if auto-assignment is on for the given address type (IPv4 or IPv6) and if the entire range falls within a managed route. + +IPv6 ranges work just like IPv4 ranges and look like this: + + { + "ipRangeStart": "fd00:feed:feed:beef:0000:0000:0000:0000", + "ipRangeEnd": "fd00:feed:feed:beef:ffff:ffff:ffff:ffff" + } + +(You can POST a shortened-form IPv6 address but the API will always report back un-shortened canonical form addresses.) + +That defines a range within network `fd00:feed:feed:beef::/64` that contains up to 2^64 addresses. If an IPv6 range is large enough, the controller will assign addresses by placing each member's device ID into the address in a manner similar to the RFC4193 and 6PLANE modes. Otherwise it will assign addresses at random. + +**Rule object format:** + +Rules are matched in order of ruleNo. If no rules match, the default action is `drop`. To allow all traffic, create a single rule with all *null* fields and an action of `accept`. + +In the future there will be many, many more types of rules. As of today only filtering by Ethernet packet type is supported. + +| Field | Type | Description | Writable | +| --------------------- | ------------- | ------------------------------------------------- | -------- | +| ruleNo | integer | Rule sorting key | YES | +| etherType | integer | Ethernet frame type (e.g. 34525 for IPv6) | YES | +| action | string | Currently either `allow` or `drop` | YES | + +**An Example: The Configuration for Earth** + +Here is an example of a correctly configured ZeroTier network with IPv4 auto-assigned addresses from 28.0.0.0/7 (a "de-facto private" space) and RFC4193 IPv6 addressing. Users might recognize this as *Earth*, our public "global LAN party" that's used for demos and testing and occasionally gaming. + +For your own networks you'll probably want to change `private` to `true` unless you like company. These rules on the other hand probably are what you want. These allow IPv4, IPv4 ARP, and IPv6 Ethernet frames. To allow only IPv4 omit the one for Ethernet type 34525 (IPv6). + + { + "nwid": "8056c2e21c000001", + "controllerInstanceId": "8ab354604debe1da27ee627c9ef94a48", + "clock": 1468004857100, + "name": "earth.zerotier.net", + "private": false, + "enableBroadcast": false, + "allowPassiveBridging": false, + "v4AssignMode": "zt", + "v6AssignMode": "rfc4193", + "multicastLimit": 64, + "creationTime": 1442292573165, + "revision": 234, + "memberRevisionCounter": 3326, + "authorizedMemberCount": 2873, + "relays": [], + "routes": [ + {"target":"28.0.0.0/7","via":null,"flags":0,"metric":0}], + "ipAssignmentPools": [ + {"ipRangeStart":"28.0.0.1","ipRangeEnd":"29.255.255.254"}], + "rules": [ + { + "ruleNo": 20, + "etherType": 2048, + "action": "accept" + },{ + "ruleNo": 21, + "etherType": 2054, + "action": "accept" + },{ + "ruleNo": 30, + "etherType": 34525, + "action": "accept" + }] + } + +#### `/controller/network/<network ID>/member` + + * Purpose: Get a set of all members on this network + * Methods: GET + * Returns: { object } + +This returns a JSON object containing all member IDs as keys and their `memberRevisionCounter` values as values. -Since Sqlite3 supports multiple processes attached to the same database, it is safe to back up a running database with the command line *sqlite3* utility: +#### `/controller/network/<network ID>/active` - sqlite3 /path/to/controller.db .dump + * Purpose: Get a set of all active members on this network + * Methods: GET + * Returns: { object } -In production ZeroTier runs this frequently and keeps many timestamped copies going back about a week. These are also backed up (encrypted) to Amazon S3 along with the rest of our data. +This returns an object containing all currently online members and the most recent `recentLog` entries for their last request. -### Administrating +#### `/controller/network/<network ID>/member/<address>` -See service/README.md for documentation on the JSON API presented by this network controller implementation. Also see *nodejs-zt1-client* for a NodeJS JavaScript interface. + * Purpose: Create, authorize, or remove a network member + * Methods: GET, POST, DELETE + * Returns: { object } -### Reliability +| Field | Type | Description | Writable | +| --------------------- | ------------- | ------------------------------------------------- | -------- | +| nwid | string | 16-digit network ID | no | +| clock | integer | Current clock, ms since epoch | no | +| address | string | Member's 10-digit ZeroTier address | no | +| authorized | boolean | Is member authorized? (for private networks) | YES | +| activeBridge | boolean | Member is able to bridge to other Ethernet nets | YES | +| identity | string | Member's public ZeroTier identity (if known) | no | +| ipAssignments | array[string] | Managed IP address assignments | YES | +| memberRevision | integer | Member revision counter | no | +| recentLog | array[object] | Recent member activity log; see below | no | -Network controllers can go offline without affecting already-configured members of running networks. You just won't be able to change anything and new members will not be able to join. +Note that managed IP assignments are only used if they fall within a managed route. Otherwise they are ignored. -High-availability can be implemented through fail-over. A simple method involves making a frequent backup of the SQLite database (use the SQLite command line client to do this safely) and the network configuration master's working directory. Then, if the master goes down, another instance of it can rapidly be provisioned elsewhere. Since ZeroTier addresses are mobile, the new instance will quickly (usually no more than 30s) take over for the old one and service requests. +**Recent log object format:** -### Limits +| Field | Type | Description | +| --------------------- | ------------- | ------------------------------------------------- | +| ts | integer | Time of request, ms since epoch | +| authorized | boolean | Was member authorized? | +| clientMajorVersion | integer | Client major version or -1 if unknown | +| clientMinorVersion | integer | Client minor version or -1 if unknown | +| clientRevision | integer | Client revision or -1 if unknown | +| fromAddr | string | Physical address if known | -A single network configuration master can administrate up to 2^24 (~16m) networks as per the ZeroTier protocol limit. There is no hard limit on the number of clients, though millions or more would impose significant CPU demands on a server. Optimizations could be implemented such as memoization/caching to reduce this. +The controller can only know a member's `fromAddr` if it's able to establish a direct path to it. Members behind very restrictive firewalls may not have this information since the controller will be receiving the member's requests by way of a relay. ZeroTier does not back-trace IP paths as packets are relayed since this would add a lot of protocol overhead. diff --git a/controller/SqliteNetworkController.cpp b/controller/SqliteNetworkController.cpp index 61ac0cd3..65051744 100644 --- a/controller/SqliteNetworkController.cpp +++ b/controller/SqliteNetworkController.cpp @@ -70,7 +70,7 @@ #define ZT_NETCONF_SQLITE_SCHEMA_VERSION_STR "4" // API version reported via JSON control plane -#define ZT_NETCONF_CONTROLLER_API_VERSION 1 +#define ZT_NETCONF_CONTROLLER_API_VERSION 2 // Number of requests to remember in member history #define ZT_NETCONF_DB_MEMBER_HISTORY_LENGTH 8 @@ -85,12 +85,13 @@ #define ZT_NETCONF_NODE_ACTIVE_THRESHOLD ((ZT_NETWORK_AUTOCONF_DELAY * 2) + 5000) // Flags for Network 'flags' field in table -#define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4 1 +#define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4_AUTO_ASSIGN 1 #define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_RFC4193 2 #define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_6PLANE 4 +#define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_AUTO_ASSIGN 8 // Flags with all V6 managed mode flags flipped off -- for masking in update operation and in string form for SQL building -#define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_MASK_S "268435449" +#define ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_MASK_S "268435441" // Uncomment to trace Sqlite for debugging //#define ZT_NETCONF_SQLITE_TRACE 1 @@ -351,7 +352,7 @@ SqliteNetworkController::SqliteNetworkController(Node *node,const char *dbPath,c if ( /* Network */ - (sqlite3_prepare_v2(_db,"SELECT name,private,enableBroadcast,allowPassiveBridging,\"flags\",multicastLimit,creationTime,revision,memberRevisionCounter FROM Network WHERE id = ?",-1,&_sGetNetworkById,(const char **)0) != SQLITE_OK) + (sqlite3_prepare_v2(_db,"SELECT name,private,enableBroadcast,allowPassiveBridging,\"flags\",multicastLimit,creationTime,revision,memberRevisionCounter,(SELECT COUNT(1) FROM Member WHERE Member.networkId = Network.id AND Member.authorized > 0) FROM Network WHERE id = ?",-1,&_sGetNetworkById,(const char **)0) != SQLITE_OK) ||(sqlite3_prepare_v2(_db,"SELECT revision FROM Network WHERE id = ?",-1,&_sGetNetworkRevision,(const char **)0) != SQLITE_OK) ||(sqlite3_prepare_v2(_db,"UPDATE Network SET revision = ? WHERE id = ?",-1,&_sSetNetworkRevision,(const char **)0) != SQLITE_OK) ||(sqlite3_prepare_v2(_db,"INSERT INTO Network (id,name,creationTime,revision) VALUES (?,?,?,1)",-1,&_sCreateNetwork,(const char **)0) != SQLITE_OK) @@ -376,8 +377,7 @@ SqliteNetworkController::SqliteNetworkController(Node *node,const char *dbPath,c ||(sqlite3_prepare_v2(_db,"DELETE FROM IpAssignmentPool WHERE networkId = ?",-1,&_sDeleteIpAssignmentPoolsForNetwork,(const char **)0) != SQLITE_OK) /* IpAssignment */ - ||(sqlite3_prepare_v2(_db,"SELECT \"type\",ip,ipNetmaskBits FROM IpAssignment WHERE networkId = ? AND (nodeId = ? OR nodeId IS NULL) AND ipVersion = ?",-1,&_sGetIpAssignmentsForNode,(const char **)0) != SQLITE_OK) - ||(sqlite3_prepare_v2(_db,"SELECT ip,ipNetmaskBits,ipVersion FROM IpAssignment WHERE networkId = ? AND nodeId = ? AND \"type\" = ? ORDER BY ip ASC",-1,&_sGetIpAssignmentsForNode2,(const char **)0) != SQLITE_OK) + ||(sqlite3_prepare_v2(_db,"SELECT ip,ipNetmaskBits,ipVersion FROM IpAssignment WHERE networkId = ? AND nodeId = ? AND \"type\" = 0 ORDER BY ip ASC",-1,&_sGetIpAssignmentsForNode,(const char **)0) != SQLITE_OK) ||(sqlite3_prepare_v2(_db,"SELECT 1 FROM IpAssignment WHERE networkId = ? AND ip = ? AND ipVersion = ? AND \"type\" = ?",-1,&_sCheckIfIpIsAllocated,(const char **)0) != SQLITE_OK) ||(sqlite3_prepare_v2(_db,"INSERT INTO IpAssignment (networkId,nodeId,\"type\",ip,ipNetmaskBits,ipVersion) VALUES (?,?,?,?,?,?)",-1,&_sAllocateIp,(const char **)0) != SQLITE_OK) ||(sqlite3_prepare_v2(_db,"DELETE FROM IpAssignment WHERE networkId = ? AND nodeId = ? AND \"type\" = ?",-1,&_sDeleteIpAllocations,(const char **)0) != SQLITE_OK) @@ -474,7 +474,6 @@ SqliteNetworkController::~SqliteNetworkController() sqlite3_finalize(_sCreateNetwork); sqlite3_finalize(_sGetNetworkRevision); sqlite3_finalize(_sSetNetworkRevision); - sqlite3_finalize(_sGetIpAssignmentsForNode2); sqlite3_finalize(_sDeleteRelaysForNetwork); sqlite3_finalize(_sCreateRelay); sqlite3_finalize(_sDeleteIpAssignmentPoolsForNetwork); @@ -740,6 +739,7 @@ NetworkController::ResultCode SqliteNetworkController::doNetworkConfigRequest(co ++nc.routeCount; } + // Assign special IPv6 addresses if these are enabled if (((network.flags & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_RFC4193) != 0)&&(nc.staticIpCount < ZT_MAX_ZT_ASSIGNED_ADDRESSES)) { nc.staticIps[nc.staticIpCount++] = InetAddress::makeIpv6rfc4193(nwid,identity.address().toInt()); nc.flags |= ZT_NETWORKCONFIG_FLAG_ENABLE_IPV6_NDP_EMULATION; @@ -749,99 +749,188 @@ NetworkController::ResultCode SqliteNetworkController::doNetworkConfigRequest(co nc.flags |= ZT_NETWORKCONFIG_FLAG_ENABLE_IPV6_NDP_EMULATION; } - if ((network.flags & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4) != 0) { - bool haveStaticIpAssignment = false; + // Get managed addresses that are assigned to this member + bool haveManagedIpv4AutoAssignment = false; + bool haveManagedIpv6AutoAssignment = false; // "special" NDP-emulated address types do not count + sqlite3_reset(_sGetIpAssignmentsForNode); + sqlite3_bind_text(_sGetIpAssignmentsForNode,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_text(_sGetIpAssignmentsForNode,2,member.nodeId,10,SQLITE_STATIC); + while (sqlite3_step(_sGetIpAssignmentsForNode) == SQLITE_ROW) { + const unsigned char *const ipbytes = (const unsigned char *)sqlite3_column_blob(_sGetIpAssignmentsForNode,0); + if ((!ipbytes)||(sqlite3_column_bytes(_sGetIpAssignmentsForNode,0) != 16)) + continue; + //const int ipNetmaskBits = sqlite3_column_int(_sGetIpAssignmentsForNode,1); + const int ipVersion = sqlite3_column_int(_sGetIpAssignmentsForNode,2); + + InetAddress ip; + if (ipVersion == 4) + ip = InetAddress(ipbytes + 12,4,0); + else if (ipVersion == 6) + ip = InetAddress(ipbytes,16,0); + else continue; + + // IP assignments are only pushed if there is a corresponding local route. We also now get the netmask bits from + // this route, ignoring the netmask bits field of the assigned IP itself. Using that was worthless and a source + // of user error / poor UX. + int routedNetmaskBits = 0; + for(unsigned int rk=0;rk<nc.routeCount;++rk) { + if ( (!nc.routes[rk].via.ss_family) && (reinterpret_cast<const InetAddress *>(&(nc.routes[rk].target))->containsAddress(ip)) ) + routedNetmaskBits = reinterpret_cast<const InetAddress *>(&(nc.routes[rk].target))->netmaskBits(); + } - sqlite3_reset(_sGetIpAssignmentsForNode); - sqlite3_bind_text(_sGetIpAssignmentsForNode,1,network.id,16,SQLITE_STATIC); - sqlite3_bind_text(_sGetIpAssignmentsForNode,2,member.nodeId,10,SQLITE_STATIC); - sqlite3_bind_int(_sGetIpAssignmentsForNode,3,4); // 4 == IPv4 - while (sqlite3_step(_sGetIpAssignmentsForNode) == SQLITE_ROW) { - const unsigned char *const ip = (const unsigned char *)sqlite3_column_blob(_sGetIpAssignmentsForNode,1); - if ((!ip)||(sqlite3_column_bytes(_sGetIpAssignmentsForNode,1) != 16)) - continue; - int ipNetmaskBits = sqlite3_column_int(_sGetIpAssignmentsForNode,2); - if ((ipNetmaskBits <= 0)||(ipNetmaskBits > 32)) + if (routedNetmaskBits > 0) { + if (nc.staticIpCount < ZT_MAX_ZT_ASSIGNED_ADDRESSES) { + ip.setPort(routedNetmaskBits); + nc.staticIps[nc.staticIpCount++] = ip; + } + if (ipVersion == 4) + haveManagedIpv4AutoAssignment = true; + else if (ipVersion == 6) + haveManagedIpv6AutoAssignment = true; + } + } + + // Auto-assign IPv6 address if auto-assignment is enabled and it's needed + if ( ((network.flags & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_AUTO_ASSIGN) != 0) && (!haveManagedIpv6AutoAssignment) && (!amActiveBridge) ) { + sqlite3_reset(_sGetIpAssignmentPools); + sqlite3_bind_text(_sGetIpAssignmentPools,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_int(_sGetIpAssignmentPools,2,6); // 6 == IPv6 + while (sqlite3_step(_sGetIpAssignmentPools) == SQLITE_ROW) { + const uint8_t *const ipRangeStartB = reinterpret_cast<const unsigned char *>(sqlite3_column_blob(_sGetIpAssignmentPools,0)); + const uint8_t *const ipRangeEndB = reinterpret_cast<const unsigned char *>(sqlite3_column_blob(_sGetIpAssignmentPools,1)); + if ((!ipRangeStartB)||(!ipRangeEndB)||(sqlite3_column_bytes(_sGetIpAssignmentPools,0) != 16)||(sqlite3_column_bytes(_sGetIpAssignmentPools,1) != 16)) continue; - if (sqlite3_column_int(_sGetIpAssignmentsForNode,0) == 0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/) { - if (nc.staticIpCount < ZT_MAX_ZT_ASSIGNED_ADDRESSES) { - struct sockaddr_in *const v4ip = reinterpret_cast<struct sockaddr_in *>(&(nc.staticIps[nc.staticIpCount++])); - v4ip->sin_family = AF_INET; - v4ip->sin_port = Utils::hton((uint16_t)ipNetmaskBits); - memcpy(&(v4ip->sin_addr.s_addr),ip + 12,4); + + uint64_t s[2],e[2],x[2],xx[2]; + memcpy(s,ipRangeStartB,16); + memcpy(e,ipRangeEndB,16); + s[0] = Utils::ntoh(s[0]); + s[1] = Utils::ntoh(s[1]); + e[0] = Utils::ntoh(e[0]); + e[1] = Utils::ntoh(e[1]); + x[0] = s[0]; + x[1] = s[1]; + + for(unsigned int trialCount=0;trialCount<1000;++trialCount) { + if ((trialCount == 0)&&(e[1] > s[1])&&((e[1] - s[1]) >= 0xffffffffffULL)) { + // First see if we can just cram a ZeroTier ID into the higher 64 bits. If so do that. + xx[0] = Utils::hton(x[0]); + xx[1] = Utils::hton(x[1] + identity.address().toInt()); + } else { + // Otherwise pick random addresses -- this technically doesn't explore the whole range if the lower 64 bit range is >= 1 but that won't matter since that would be huge anyway + Utils::getSecureRandom((void *)xx,16); + if ((e[0] > s[0])) + xx[0] %= (e[0] - s[0]); + else xx[0] = 0; + if ((e[1] > s[1])) + xx[1] %= (e[1] - s[1]); + else xx[1] = 0; + xx[0] = Utils::hton(x[0] + xx[0]); + xx[1] = Utils::hton(x[1] + xx[1]); + } + + InetAddress ip6((const void *)xx,16,0); + + // Check if this IP is within a local-to-Ethernet routed network + int routedNetmaskBits = 0; + for(unsigned int rk=0;rk<nc.routeCount;++rk) { + if ( (!nc.routes[rk].via.ss_family) && (nc.routes[rk].target.ss_family == AF_INET6) && (reinterpret_cast<const InetAddress *>(&(nc.routes[rk].target))->containsAddress(ip6)) ) + routedNetmaskBits = reinterpret_cast<const InetAddress *>(&(nc.routes[rk].target))->netmaskBits(); + } + + // If it's routed, then try to claim and assign it and if successful end loop + if (routedNetmaskBits > 0) { + sqlite3_reset(_sCheckIfIpIsAllocated); + sqlite3_bind_text(_sCheckIfIpIsAllocated,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_blob(_sCheckIfIpIsAllocated,2,(const void *)ip6.rawIpData(),16,SQLITE_STATIC); + sqlite3_bind_int(_sCheckIfIpIsAllocated,3,6); // 6 == IPv6 + sqlite3_bind_int(_sCheckIfIpIsAllocated,4,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); + if (sqlite3_step(_sCheckIfIpIsAllocated) != SQLITE_ROW) { + // No rows returned, so the IP is available + sqlite3_reset(_sAllocateIp); + sqlite3_bind_text(_sAllocateIp,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_text(_sAllocateIp,2,member.nodeId,10,SQLITE_STATIC); + sqlite3_bind_int(_sAllocateIp,3,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); + sqlite3_bind_blob(_sAllocateIp,4,(const void *)ip6.rawIpData(),16,SQLITE_STATIC); + sqlite3_bind_int(_sAllocateIp,5,routedNetmaskBits); // IP netmask bits from matching route + sqlite3_bind_int(_sAllocateIp,6,6); // 6 == IPv6 + if (sqlite3_step(_sAllocateIp) == SQLITE_DONE) { + ip6.setPort(routedNetmaskBits); + if (nc.staticIpCount < ZT_MAX_ZT_ASSIGNED_ADDRESSES) + nc.staticIps[nc.staticIpCount++] = ip6; + break; + } + } } - haveStaticIpAssignment = true; } } + } - if ((!haveStaticIpAssignment)&&(!amActiveBridge)) { - // Attempt to auto-assign an IPv4 address from an available routed pool if there is one - sqlite3_reset(_sGetIpAssignmentPools); - sqlite3_bind_text(_sGetIpAssignmentPools,1,network.id,16,SQLITE_STATIC); - sqlite3_bind_int(_sGetIpAssignmentPools,2,4); // 4 == IPv4 - - while (sqlite3_step(_sGetIpAssignmentPools) == SQLITE_ROW) { - const unsigned char *ipRangeStartB = reinterpret_cast<const unsigned char *>(sqlite3_column_blob(_sGetIpAssignmentPools,0)); - const unsigned char *ipRangeEndB = reinterpret_cast<const unsigned char *>(sqlite3_column_blob(_sGetIpAssignmentPools,1)); - if ((!ipRangeStartB)||(!ipRangeEndB)||(sqlite3_column_bytes(_sGetIpAssignmentPools,0) != 16)||(sqlite3_column_bytes(_sGetIpAssignmentPools,1) != 16)) - continue; + // Auto-assign IPv4 address if auto-assignment is enabled and it's needed + if ( ((network.flags & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4_AUTO_ASSIGN) != 0) && (!haveManagedIpv4AutoAssignment) && (!amActiveBridge) ) { + sqlite3_reset(_sGetIpAssignmentPools); + sqlite3_bind_text(_sGetIpAssignmentPools,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_int(_sGetIpAssignmentPools,2,4); // 4 == IPv4 + while (sqlite3_step(_sGetIpAssignmentPools) == SQLITE_ROW) { + const unsigned char *ipRangeStartB = reinterpret_cast<const unsigned char *>(sqlite3_column_blob(_sGetIpAssignmentPools,0)); + const unsigned char *ipRangeEndB = reinterpret_cast<const unsigned char *>(sqlite3_column_blob(_sGetIpAssignmentPools,1)); + if ((!ipRangeStartB)||(!ipRangeEndB)||(sqlite3_column_bytes(_sGetIpAssignmentPools,0) != 16)||(sqlite3_column_bytes(_sGetIpAssignmentPools,1) != 16)) + continue; - uint32_t ipRangeStart = Utils::ntoh(*(reinterpret_cast<const uint32_t *>(ipRangeStartB + 12))); - uint32_t ipRangeEnd = Utils::ntoh(*(reinterpret_cast<const uint32_t *>(ipRangeEndB + 12))); - if ((ipRangeEnd <= ipRangeStart)||(ipRangeStart == 0)) - continue; - uint32_t ipRangeLen = ipRangeEnd - ipRangeStart; - - // Start with the LSB of the member's address - uint32_t ipTrialCounter = (uint32_t)(identity.address().toInt() & 0xffffffff); - - for(uint32_t k=ipRangeStart,l=0;(k<=ipRangeEnd)&&(l < 1000000);++k,++l) { - uint32_t ip = (ipRangeLen > 0) ? (ipRangeStart + (ipTrialCounter % ipRangeLen)) : ipRangeStart; - ++ipTrialCounter; - if ((ip & 0x000000ff) == 0x000000ff) - continue; // don't allow addresses that end in .255 - - // Check if this IPv4 IP is within a local-to-Ethernet routed network - int routedNetmaskBits = 0; - for(unsigned int rk=0;rk<nc.routeCount;++rk) { - if ((!nc.routes[rk].via.ss_family)&&(nc.routes[rk].target.ss_family == AF_INET)) { - uint32_t targetIp = Utils::ntoh((uint32_t)(reinterpret_cast<const struct sockaddr_in *>(&(nc.routes[rk].target))->sin_addr.s_addr)); - int targetBits = Utils::ntoh((uint16_t)(reinterpret_cast<const struct sockaddr_in *>(&(nc.routes[rk].target))->sin_port)); - if ((ip & (0xffffffff << (32 - targetBits))) == targetIp) { - routedNetmaskBits = targetBits; - break; - } + uint32_t ipRangeStart = Utils::ntoh(*(reinterpret_cast<const uint32_t *>(ipRangeStartB + 12))); + uint32_t ipRangeEnd = Utils::ntoh(*(reinterpret_cast<const uint32_t *>(ipRangeEndB + 12))); + if ((ipRangeEnd <= ipRangeStart)||(ipRangeStart == 0)) + continue; + uint32_t ipRangeLen = ipRangeEnd - ipRangeStart; + + // Start with the LSB of the member's address + uint32_t ipTrialCounter = (uint32_t)(identity.address().toInt() & 0xffffffff); + + for(uint32_t k=ipRangeStart,trialCount=0;(k<=ipRangeEnd)&&(trialCount < 1000);++k,++trialCount) { + uint32_t ip = (ipRangeLen > 0) ? (ipRangeStart + (ipTrialCounter % ipRangeLen)) : ipRangeStart; + ++ipTrialCounter; + if ((ip & 0x000000ff) == 0x000000ff) + continue; // don't allow addresses that end in .255 + + // Check if this IP is within a local-to-Ethernet routed network + int routedNetmaskBits = 0; + for(unsigned int rk=0;rk<nc.routeCount;++rk) { + if ((!nc.routes[rk].via.ss_family)&&(nc.routes[rk].target.ss_family == AF_INET)) { + uint32_t targetIp = Utils::ntoh((uint32_t)(reinterpret_cast<const struct sockaddr_in *>(&(nc.routes[rk].target))->sin_addr.s_addr)); + int targetBits = Utils::ntoh((uint16_t)(reinterpret_cast<const struct sockaddr_in *>(&(nc.routes[rk].target))->sin_port)); + if ((ip & (0xffffffff << (32 - targetBits))) == targetIp) { + routedNetmaskBits = targetBits; + break; } } + } - // If it's routed, then try to claim and assign it and if successful end loop - if (routedNetmaskBits > 0) { - uint32_t ipBlob[4]; // actually a 16-byte blob, we put IPv4s in the last 4 bytes - ipBlob[0] = 0; ipBlob[1] = 0; ipBlob[2] = 0; ipBlob[3] = Utils::hton(ip); - sqlite3_reset(_sCheckIfIpIsAllocated); - sqlite3_bind_text(_sCheckIfIpIsAllocated,1,network.id,16,SQLITE_STATIC); - sqlite3_bind_blob(_sCheckIfIpIsAllocated,2,(const void *)ipBlob,16,SQLITE_STATIC); - sqlite3_bind_int(_sCheckIfIpIsAllocated,3,4); // 4 == IPv4 - sqlite3_bind_int(_sCheckIfIpIsAllocated,4,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); - if (sqlite3_step(_sCheckIfIpIsAllocated) != SQLITE_ROW) { - // No rows returned, so the IP is available - sqlite3_reset(_sAllocateIp); - sqlite3_bind_text(_sAllocateIp,1,network.id,16,SQLITE_STATIC); - sqlite3_bind_text(_sAllocateIp,2,member.nodeId,10,SQLITE_STATIC); - sqlite3_bind_int(_sAllocateIp,3,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); - sqlite3_bind_blob(_sAllocateIp,4,(const void *)ipBlob,16,SQLITE_STATIC); - sqlite3_bind_int(_sAllocateIp,5,routedNetmaskBits); // IP netmask bits from matching route - sqlite3_bind_int(_sAllocateIp,6,4); // 4 == IPv4 - if (sqlite3_step(_sAllocateIp) == SQLITE_DONE) { - if (nc.staticIpCount < ZT_MAX_ZT_ASSIGNED_ADDRESSES) { - struct sockaddr_in *const v4ip = reinterpret_cast<struct sockaddr_in *>(&(nc.staticIps[nc.staticIpCount++])); - v4ip->sin_family = AF_INET; - v4ip->sin_port = Utils::hton((uint16_t)routedNetmaskBits); - v4ip->sin_addr.s_addr = Utils::hton(ip); - } - haveStaticIpAssignment = true; - break; + // If it's routed, then try to claim and assign it and if successful end loop + if (routedNetmaskBits > 0) { + uint32_t ipBlob[4]; // actually a 16-byte blob, we put IPv4s in the last 4 bytes + ipBlob[0] = 0; ipBlob[1] = 0; ipBlob[2] = 0; ipBlob[3] = Utils::hton(ip); + sqlite3_reset(_sCheckIfIpIsAllocated); + sqlite3_bind_text(_sCheckIfIpIsAllocated,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_blob(_sCheckIfIpIsAllocated,2,(const void *)ipBlob,16,SQLITE_STATIC); + sqlite3_bind_int(_sCheckIfIpIsAllocated,3,4); // 4 == IPv4 + sqlite3_bind_int(_sCheckIfIpIsAllocated,4,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); + if (sqlite3_step(_sCheckIfIpIsAllocated) != SQLITE_ROW) { + // No rows returned, so the IP is available + sqlite3_reset(_sAllocateIp); + sqlite3_bind_text(_sAllocateIp,1,network.id,16,SQLITE_STATIC); + sqlite3_bind_text(_sAllocateIp,2,member.nodeId,10,SQLITE_STATIC); + sqlite3_bind_int(_sAllocateIp,3,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); + sqlite3_bind_blob(_sAllocateIp,4,(const void *)ipBlob,16,SQLITE_STATIC); + sqlite3_bind_int(_sAllocateIp,5,routedNetmaskBits); // IP netmask bits from matching route + sqlite3_bind_int(_sAllocateIp,6,4); // 4 == IPv4 + if (sqlite3_step(_sAllocateIp) == SQLITE_DONE) { + if (nc.staticIpCount < ZT_MAX_ZT_ASSIGNED_ADDRESSES) { + struct sockaddr_in *const v4ip = reinterpret_cast<struct sockaddr_in *>(&(nc.staticIps[nc.staticIpCount++])); + v4ip->sin_family = AF_INET; + v4ip->sin_port = Utils::hton((uint16_t)routedNetmaskBits); + v4ip->sin_addr.s_addr = Utils::hton(ip); } + break; } } } @@ -997,7 +1086,7 @@ unsigned int SqliteNetworkController::handleControlPlaneHttpPOST( sqlite3_bind_text(_sAllocateIp,2,addrs,10,SQLITE_STATIC); sqlite3_bind_int(_sAllocateIp,3,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); sqlite3_bind_blob(_sAllocateIp,4,(const void *)ipBlob,16,SQLITE_STATIC); - sqlite3_bind_int(_sAllocateIp,5,(int)a.netmaskBits()); + sqlite3_bind_int(_sAllocateIp,5,(int)a.netmaskBits()); // NOTE: this field is now ignored but set it anyway sqlite3_bind_int(_sAllocateIp,6,ipVersion); if (sqlite3_step(_sAllocateIp) != SQLITE_DONE) return 500; @@ -1181,10 +1270,10 @@ unsigned int SqliteNetworkController::handleControlPlaneHttpPOST( } else if (!strcmp(j->u.object.values[k].name,"v4AssignMode")) { if ((j->u.object.values[k].value->type == json_string)&&(!strcmp(j->u.object.values[k].value->u.string.ptr,"zt"))) { if (sqlite3_prepare_v2(_db,"UPDATE Network SET \"flags\" = (\"flags\" | ?) WHERE id = ?",-1,&stmt,(const char **)0) == SQLITE_OK) - sqlite3_bind_int(stmt,1,(int)ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4); + sqlite3_bind_int(stmt,1,(int)ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4_AUTO_ASSIGN); } else { if (sqlite3_prepare_v2(_db,"UPDATE Network SET \"flags\" = (\"flags\" & ?) WHERE id = ?",-1,&stmt,(const char **)0) == SQLITE_OK) - sqlite3_bind_int(stmt,1,(int)(ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4 ^ 0xfffffff)); + sqlite3_bind_int(stmt,1,(int)(ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4_AUTO_ASSIGN ^ 0xfffffff)); } } else if (!strcmp(j->u.object.values[k].name,"v6AssignMode")) { int fl = 0; @@ -1195,6 +1284,8 @@ unsigned int SqliteNetworkController::handleControlPlaneHttpPOST( fl |= ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_RFC4193; else if (!strcmp(f,"6plane")) fl |= ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_6PLANE; + else if (!strcmp(f,"zt")) + fl |= ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_AUTO_ASSIGN; } } if (sqlite3_prepare_v2(_db,"UPDATE Network SET \"flags\" = ((\"flags\" & " ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_MASK_S ") | ?) WHERE id = ?",-1,&stmt,(const char **)0) == SQLITE_OK) @@ -1680,22 +1771,21 @@ unsigned int SqliteNetworkController::_doCPGet( _jsonEscape(memberIdStr).c_str()); responseBody = json; - sqlite3_reset(_sGetIpAssignmentsForNode2); - sqlite3_bind_text(_sGetIpAssignmentsForNode2,1,nwids,16,SQLITE_STATIC); - sqlite3_bind_text(_sGetIpAssignmentsForNode2,2,addrs,10,SQLITE_STATIC); - sqlite3_bind_int(_sGetIpAssignmentsForNode2,3,(int)0 /*ZT_IP_ASSIGNMENT_TYPE_ADDRESS*/); + sqlite3_reset(_sGetIpAssignmentsForNode); + sqlite3_bind_text(_sGetIpAssignmentsForNode,1,nwids,16,SQLITE_STATIC); + sqlite3_bind_text(_sGetIpAssignmentsForNode,2,addrs,10,SQLITE_STATIC); bool firstIp = true; - while (sqlite3_step(_sGetIpAssignmentsForNode2) == SQLITE_ROW) { - int ipversion = sqlite3_column_int(_sGetIpAssignmentsForNode2,2); + while (sqlite3_step(_sGetIpAssignmentsForNode) == SQLITE_ROW) { + int ipversion = sqlite3_column_int(_sGetIpAssignmentsForNode,2); char ipBlob[16]; - memcpy(ipBlob,(const void *)sqlite3_column_blob(_sGetIpAssignmentsForNode2,0),16); + memcpy(ipBlob,(const void *)sqlite3_column_blob(_sGetIpAssignmentsForNode,0),16); InetAddress ip( (const void *)(ipversion == 6 ? ipBlob : &ipBlob[12]), (ipversion == 6 ? 16 : 4), - (unsigned int)sqlite3_column_int(_sGetIpAssignmentsForNode2,1) + (unsigned int)sqlite3_column_int(_sGetIpAssignmentsForNode,1) ); responseBody.append(firstIp ? "\"" : ",\""); - responseBody.append(_jsonEscape(ip.toString())); + responseBody.append(_jsonEscape(ip.toIpString())); responseBody.push_back('"'); firstIp = false; } @@ -1801,8 +1891,11 @@ unsigned int SqliteNetworkController::_doCPGet( v6modes.push_back(','); v6modes.append("6plane"); } - if (v6modes.length() == 0) - v6modes = "none"; + if ((fl & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V6_AUTO_ASSIGN) != 0) { + if (v6modes.length() > 0) + v6modes.push_back(','); + v6modes.append("zt"); + } Utils::snprintf(json,sizeof(json), "{\n" @@ -1828,13 +1921,13 @@ unsigned int SqliteNetworkController::_doCPGet( (sqlite3_column_int(_sGetNetworkById,1) > 0) ? "true" : "false", (sqlite3_column_int(_sGetNetworkById,2) > 0) ? "true" : "false", (sqlite3_column_int(_sGetNetworkById,3) > 0) ? "true" : "false", - (((fl & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4) != 0) ? "zt" : "none"), + (((fl & ZT_DB_NETWORK_FLAG_ZT_MANAGED_V4_AUTO_ASSIGN) != 0) ? "zt" : ""), v6modes.c_str(), sqlite3_column_int(_sGetNetworkById,5), (unsigned long long)sqlite3_column_int64(_sGetNetworkById,6), (unsigned long long)sqlite3_column_int64(_sGetNetworkById,7), (unsigned long long)sqlite3_column_int64(_sGetNetworkById,8), - (unsigned long long)0); // TODO: authorized member count, but more efficiently + (unsigned long long)sqlite3_column_int64(_sGetNetworkById,9)); responseBody = json; sqlite3_reset(_sGetRelays); diff --git a/controller/SqliteNetworkController.hpp b/controller/SqliteNetworkController.hpp index efeb63b1..145788c7 100644 --- a/controller/SqliteNetworkController.hpp +++ b/controller/SqliteNetworkController.hpp @@ -154,7 +154,6 @@ private: sqlite3_stmt *_sCreateNetwork; sqlite3_stmt *_sGetNetworkRevision; sqlite3_stmt *_sSetNetworkRevision; - sqlite3_stmt *_sGetIpAssignmentsForNode2; sqlite3_stmt *_sDeleteRelaysForNetwork; sqlite3_stmt *_sCreateRelay; sqlite3_stmt *_sDeleteIpAssignmentPoolsForNetwork; |