Selenium + Python + Tor = ❤

So I've been trying Selenium, a browser automation tool, to see what you can do with browser manipulation and to not get caught doing so I came to the idea to use Tor Browser.

If you don't know Tor it's a browser based on the Firefox engine that proxies all your connection through a network of relays (you can find out more on their website).

You can configure Selenium to use any popular browser (Chrome, Firefox, Safari, etc.) but you cannot choose Tor so how can we accomplish that?
There are two options which I will both be covered in this post:

1. The tor-browser-selenium package
2. Profile options

Prerequisites

• an up-to-date Firefox browser (you can also use any other browser since the code will be mostly the same for any browser)
• selenium (version 4.0.0.b4 or greater)
• geckodriver (or any driver fitting your desired browser)
• python3
• Tor

1 – The tor-browser-selenium package

Be aware: at the time writing this (June 20, 2021) the latest release is v0.5.2

To utilize this package you only need to follow 3 steps:

1. Install the package using pip pip install tbselenium
2. Import the package from tbselenium.tbdriver import TorBrowserDriver
3. Copy these two lines into your code:

with TorBrowserDriver("/path/to/TorBrowserBundle/") as driver:
    driver.get('https://check.torproject.org')

Your code should now look something like this:

from tbselenium.tbdriver import TorBrowserDriver
with TorBrowserDriver("/path/to/TorBrowserBundle/") as driver:
    driver.get('https://check.torproject.org')

The package can accomplish more than this and there are more usage options available. You can read more at the GitHub repo.

2 – Setting profile options

Be aware: I only tested this using Firefox but this should also work for any other browser, if it doesn't just add Kokosnuss.exe#1478 on Discord or write us a mail to konstantinw[at]ledbrain.de

Since we are using selenium >v4, profiles are deprecated so many of the implementations you find won't work anymore. The key is to not use the Tor browser directly but rather just use the proxy the browser provides which is done is some easy steps:

1. Import the Options class: from selenium.webdriver.firefox.options import Options (change firefox to your browser accordingly)
2. Copy these lines into your code:

options = Options()
options.set_preference('network.proxy.type', 1)
options.set_preference('network.proxy.socks', '127.0.0.1')
options.set_preference('network.proxy.socks_port', 9150)
options.set_preference('network.proxy.socks_remote_dns', False)

Line 1 creates a new options object
Line 2 tell Firefox we configure the proxy manually
Line 3 sets the socket to localhost (where Tor is running)
Line 4 sets the sockets port (this can vary, if it does not work, try 9050)
Line 5 tells Firefox to perform DNS lookups on the client (not the proxy server)

When using Chrome it might be enough to set the following (I did not test this, but it might work just fine):

options = Options()
PROXY = "socks5://localhost:9050" # IP:PORT or HOST:PORT
options.add_argument('--proxy-server=%s' % PROXY)

Either way, don't forget to tell the driver constructor to use these options which might look like this depending on your browser:

# ... the code from above
with webdriver.Firefox(options=options) as driver:
# ... the rest of your code

Testing the config

You may test your code is running over the Tor proxy by opening https://check.torproject.org just use driver.get(https://check.torproject.org) (if you want to be fancy take a screenshot)

Conclusion

It's possible to use Tor and sometimes you should use it if you are doing some more or less sketchy browser manipulation stuff. Take the code from above, copy and paste it into yours and see all the Onion magic happen.
Big thanks to these two StackOverflow posts from Chirag Shetty and DebanjanB which really helped me figure this out. And to Jan for helping me fixing bugs and motivating me!

Happy coding and safe browsing!