Deanonymizing Tor Circuits

Since starting my Tor onion service that provides access to the Internet Archive, I've seen a wide range of Tor-based attacks. I have documented many of these in various blog entries. For example:

• Attacked Over Tor: 3 bad bots, 1 aggressive crawler, and 1 attack bot.
• Stopping Tor Attacks: Mitigation options for aggressive, abusive, and DDoS bots.
• Tor Attacks Revisited : Lots of different types of attacks that my server has encountered over the Tor connection.
• A New Tor Attack: An updated solution for an updated attack.

Over the last month, I've noticed a new type of attack. It took me a while to figure out what they are trying to do: they appear to be trying to map out part of the Tor circuit used by my hidden service.

(In this blog entry, you'll see Tor spelled "Tor" and "tor". Uppercase is the protocol's name, while lowercase is the name of the program.)

Typical Usage

In order to identify attacks, you must first identify typical usage. Unfortunately, even in full debug mode, the basic tor-daemon source code does not provide enough information. I've modified my copy of the source code to be much more informative. For example, my tor daemon logs every rendezvous point. Basically, I edited src/or/rendservice.c and added a print statement to the function rend_service_receive_introduction:

tor_assert(launched->build_state);



  /** Neal: Log rendezvous point **/

  log_warn(LD_REND,"Rendezvous [%s]", safe_str_client(extend_info_describe(rp)));


  /* Fill in the circuit's state. */

(And before anyone asks: No, this does not violate your privacy. The rendezvous is an intermediary node located outside of the initial circuits established by both the browser and the service. And yes, logs are automatically deleted after a week.)

With typical usage, I'll see the tor daemon connect to a rendezvous point, and then a moment later there will be a bunch of HTTP connections to the server. For example, this is me connecting to my server over Tor with the Tor Browser:

Jan 30 09:26:51 Tor[24247]: Rendezvous [$F5746F6257DFE87E3A90753C2A0439926C55552F~$F5746F6257DFE87E3A at 82.169.130.61]

[30/Jan/2020:09:26:54 -0700] "archivecrfip2lpi.onion" 200 10301 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

[30/Jan/2020:09:26:54 -0700] "archivecrfip2lpi.onion" 200 7312 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

[30/Jan/2020:09:26:54 -0700] "archivecrfip2lpi.onion" 200 6492 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

[30/Jan/2020:09:26:54 -0700] "archivecrfip2lpi.onion" 200 3827 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

The IP address from the rendezvous point is always a Tor node, and the hex values before it is the Tor node's unique fingerprint. This is public and published information, and you can look it up at metrics.torproject.org . In this case, the rendezvous node has the nickname "Hijnn", it exists at 82.169.130.61, and when I wrote this, it had been up for 8 days 21 hours 32 minutes and 47 seconds.

With Tor, the entry node (guard) can be publicly listed or kept unlisted. (The unlisted guard nodes are usually reserved as bridges -- unlisted to prevent IP-based censorship.) However, the relay, rendezvous, and exit nodes must be publicly known so that lots of Tor traffic will use them. This is part of Tor's anonymity: your traffic is indistinguishable from everyone else.

Note: Technically, you can run your own private exit node, but this defeats the purpose. If you're the only person using your private exit node, then network activity can be attributed to you. Similarly, the rendezvous node should be a publicly known service so that traffic can commingle for anonymity. You don't want to use your own private rendezvous node since that makes your activity stand out.

Typical Usage vs Bots

A lot of bots have atypical usage. For example, many bots will only do one HTTP connection at a time (single threaded). And the badly written bots will perform one HTTP request per rendezvous negotiation.

However, aggressive bots can be spotted because they negotiate multiple rendezvous nodes before sending through a large number of HTTP requests. For example:

Jan 30 14:19:22 Tor[17199]: Rendezvous [$1B710612CB33CA26B7CF9964DFE79E60B45FAF60~$1B710612CB33CA26B7 at 35.228.99.44]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$BFC1305C8B37E5161C2E37135DF2D4E53CC38ACE~$BFC1305C8B37E5161C at 188.68.46.164]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$53134D9637D9FBE565FA1E3AF82B23CC964C56D6~$53134D9637D9FBE565 at 37.59.76.255]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$41EEC4CFA01E8982643F1AF3CD84315329D2B58E~$41EEC4CFA01E898264 at 95.211.147.99]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$2A621A40FF3081F612946FDFB8DC781BCE859A05~$2A621A40FF3081F612 at 116.203.88.24]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$861BCFDD148973985E7FE97C7455C9E4AC4E13BE~$861BCFDD148973985E at 148.251.22.104]

Jan 30 14:19:22 Tor[17199]: Rendezvous: Circuit closing

Jan 30 14:19:22 Tor[17199]: Rendezvous [$C0E6A667064385B9CB5A685CEB06B85EDDA6AA00~$C0E6A667064385B9CB at 77.123.155.45]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$204ECC4FF8F93862E82FA19C53B5BC98B1AF6046~$204ECC4FF8F93862E8 at 54.37.207.84]

Jan 30 14:19:22 Tor[17199]: Rendezvous: Circuit closing

Jan 30 14:19:22 Tor[17199]: Rendezvous [$964B4E8A75263A69769541F2764563DABDD995D2~$964B4E8A75263A6976 at 68.67.32.31]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$F1FE9BEF7DE30B2BA1547270495A6CE662C9C9E5~$F1FE9BEF7DE30B2BA1 at 83.162.159.206]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$E2CF09F998248C71139B24B2C92740AEDB1C6D2A~$E2CF09F998248C7113 at 107.180.239.164]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$75A931404453030821C547A4FAA9094A06C48C7A~$75A931404453030821 at 46.101.183.160]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$A0547D9D5383B4A6314CBAF3006EAECA197CD82F~$A0547D9D5383B4A631 at 148.251.137.3]

[30/Jan/2020:14:19:22 -0700] "web.archivecrfip2lpi.onion" 404 7782 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

Jan 30 14:19:22 Tor[17199]: Rendezvous [$CE1FD7659F2DFE92B883083C0C6C974616D17F3D~$CE1FD7659F2DFE92B8 at 185.15.72.62]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$24B1DF80768332990BA8F1230BA41D95D82D52BD~$24B1DF80768332990B at 81.7.10.86]

Jan 30 14:19:22 Tor[17199]: Rendezvous: Circuit closing

Jan 30 14:19:22 Tor[17199]: Rendezvous: Circuit closing

Jan 30 14:19:22 Tor[17199]: Rendezvous [$6940247E04C839D268543E7F62566A91E40567E3~$6940247E04C839D268 at 176.9.57.152]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$E95955CD7AB012DE770711878F147C784FC13D37~$E95955CD7AB012DE77 at 164.132.226.30]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$24F97F98C45E4754655BE66799049763DAEE99CE~$24F97F98C45E475465 at 136.243.4.139]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$B630BE802A803403F4BBEDF1C4B7BE7B31A89305~$B630BE802A803403F4 at 212.51.159.148]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$00D2269DBC1A39D137160789C7B614197DB30C70~$00D2269DBC1A39D137 at 51.15.97.42]

Jan 30 14:19:22 Tor[17199]: Rendezvous [$11C9529C9D0671545EAEF80DFE209AD977BCE908~$11C9529C9D0671545E at 95.91.4.56]

Jan 30 14:19:22 Tor[17199]: Rendezvous: Circuit closing

[30/Jan/2020:14:19:23 -0700] "web.archivecrfip2lpi.onion" 404 7782 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

[30/Jan/2020:14:19:23 -0700] "web.archivecrfip2lpi.onion" 404 7872 "" "Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Firefox/68.0"

Jan 30 14:19:23 Tor[17199]: message repeated 9 times: [ Rendezvous: Circuit closing]

Jan 30 14:19:23 Tor[17199]: Rendezvous: Circuit closing

All of these rendezvous points are known Tor nodes. This is from an agressive crawler that spawns lots of crawling processes at the same time. (The "circuit closing" messages denote my server detecting the crawler and shutting it down before the first GET request. My detection has zero false positives. How you establish connections over Tor is a distinct and profileable attribute. You are not anonymous.)

Forward in Reverse

Back in 2017, I mentioned oneodd attacker. (I don't know exactly what he's doing. But since he's not a regular user, I'm classifying it as an attack.) He would specify the rendezvous address, but it would be in the wrong endian (wrong byte order). For example, 95.216.53.157 is a known Tor node, but this bot would request a rendezvous using the numbers in reverse: 157.53.216.95. The reversed address is not a know Tor node. Even the server fingerprints were wrong. As a result, the connection requests would fail, but that didn't stop him from trying over and over.

This went on for years. Oddly, sometimes the connection would succeed. My suspicion is that he was running some hostile nodes. If he saw the reverse address or wrong fingerprint, then he would correct it. This could be used as a flag to track the traffic volume or identify the last node in my server's Tor circuit.

Private Address Bot

Late last year, the reverse-address bot stopped. It was replaced by (what I'm calling) the private-address bot. Here's a few examples:

Jan 30 07:35:59 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:01 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:03 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:04 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:06 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:08 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:09 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:10 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]

Jan 30 07:36:11 Tor[24247]: Rendezvous [$6B4108C2ACE1A805173B756A138A50C8770DAD2F~$6B4108C2ACE1A80517 at 149.248.4.19]



Jan 30 09:35:11 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:20 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:21 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:21 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:22 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:23 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:23 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:24 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:25 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]

Jan 30 09:35:26 Tor[24247]: Rendezvous [$7285A9997A37F9BB39C8007FA975331DE84F48A1~$7285A9997A37F9BB39 at 139.180.198.17]



Jan 30 13:02:24 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:24 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:25 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:27 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:27 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:28 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:29 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:30 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:31 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

Jan 30 13:02:31 Tor[17199]: Rendezvous [$7555AE0CF28142B4BF82E3BDAF006338D72C1123~$7555AE0CF28142B4BF at 45.76.134.212]

There are usually 8-10 requests at a time, and these clusters repeat a few times per hour. The IP addresses are not random; I'll see the same IPs come through in batches. According to metrics.torproject.org , the IP addresses and fingerprints are not known Tor nodes; they don't even show up in my tor daemon's cache of known descriptors. However, these IPs are running Tor and have the OR port open (9001/tcp).

Unlike the reverse-address bot, which provided bogus addresses, these IPs are all known cloud providers. (Usually Choopa LLC -- a cloud provider that is regularly used by hostile actors .)

As far as I can tell, this developer is shooting out packets and waiting to see which Tor node responds. This way, they know that my onion service uses this specific node in the Tor relay.

If you use the Tor Browser, then you can view your Tor circuit (path through the Tor network). For example:

The typical path between the browser and Tor onion service has 7 hops: from my browser to my guard (in this example, located in the UK) and through two relays (France and Germany). At the same time, the onion service has a guard and two relays that are unknown to the Tor Browser. (The image shows "Relay, Relay Relay", but that bottom Relay is the service's guard.) What you are seeing are two Tor circuits, one from the browser out, and one from the service in, that meet in the middle.

So what is this attacker doing? I think he's trying to map out the next unknown relay! And since the circuit changes every few minutes, he's repeatedly mapping out the current "last relay" used by my onion service.

Chaining Exploits

Identifying the last node in my circuit does not tell him where I am located. However, this does provide useful information to the attacker. (What is he doing? He's trying to find my guard node!)

Some Tor nodes are part of registered families. That is, a bunch of Tor nodes that are all run by the same organization. For example, niftyspinymouse ( 5.196.213.57 ) is part of a large family of related Tor nodes. (As I am writing this, there are currently 68 active nodes in this family. This screenshot just shows some of them.)

The Tor guard node rarely changes, even if the rest of the circuit changes often. Moreover, the tor daemon will never make a circuit using two nodes that are part of the same known family. So if he sees that my route uses niftyspinymouse, then he can immediate determine that my guard node is not any of these 68 Tor nodes.

Remember: the guard rarely changes but the other two hops change often. If he can repeatedly map out my circuit's last node, then he can build a large exclusion list. If he can exclude everything else, then he can find my guard node. And if he can't exclude everything, then he can probably whittle it down to a handful of possible guard nodes.

Identifying the possible guard node does not tell him my server's real address. However, there's a known attack where an attacker DDoS's the guard node. This will disable the guard and temporarily knock my service offline -- until my service renegotiates with a new guard. But the guard still doesn't identify where my server is located.

However, there's a second attack. The attacker can run one or more hostile guard nodes. If he can knock me off enough guards, my tor daemon will eventually choose one of his guards. Then he can identify my actual network address and directly attack my server. (This happened to me once.)

Alternately, the tor daemon tracks bad nodes. If he can get my tor daemon to mark enough nodes as bad, then he can knock my service offline because the tor process won't be able to connect to any guards. (And yes, that's happened before. That was one of the nastier outages that my onion service experienced.)

Mitigating Attacks

There are a few mitigation options here. (I'm using them, but I doubt other people are doing this. Each requires some serious programming. Unfortunately, the Tor Project has been less than open to implementing any kind of mitigation option.)

The first option is to use the torrc's ExcludeExitNodes to exclude entire countries (e.g., ExcludeExitNodes {br},{ru},{pl} ) along with " StrictNodes 1 ". This tells my tor daemon to never intentionally connect to nodes in certain countries. While trying to identify my guard node, the attacker can rule out much of the Tor network. However, there are always at least a few hundred possible places where my guard could be located -- and that is assuming that I'm not using a bridge. (My modified tor daemon randomly excludes different countries each time it chooses a new guard. The Tor Project should seriously consider making this a configuration option for onion servers.)

The second option is to change the server's IP address. This way, even if they find your address, you become a moving target. With IPv4, you may be stuck with a fixed address because there are so few addresses available. However, if you use IPv6, then this is relatively easy to setup. I mean, sure, if the attacker somehow becomes my guard, then he knows my IPv6 address and can quickly narrow my subnet down to a /64 or /92 range. But there are still millions of addresses for my server to use. Each time I choose a new guard, I can choose a new address. (If only Tor had better IPv6 support. Right Joe and IPv6Sec ? Right now, only about 15% of Tor nodes support IPv6, and you can't run an IPv6-only Tor node.)

Finally, there is a third option: Every tor daemon downloads the list of known public nodes and stores it locally while it is running. (See $HOME/.tor/cached-microdescs* ) And the rendezvous point must be in this list (because you shouldn't have a private rendezvous node). If the tor daemon checked to see if the rendezvous node was known before attempting to connect to it, then this attack would completely fail. (Seriously, Tor Project. Consider adding this.)

(Too bad ExcludeNodes doesn't support ASNs. Otherwise, I would suggest blacklisting all of Choopa.)

If you run any kind of service on the plain old Internet, then you are bound to see blind scans and generic attacks. WordPress exploits, SQL injections, and scans for the latest vulnerabilities are common. Once in a blue moon, you might see a specific and directed attack. But on Tor? I often see more attacks and badly behaved bots than regular users. And very few are drive-by generic attacks. From a researcher viewpoint, I see some of the most creative attacks over Tor.