In our previous post, we talked about the Dark Web and how accessing the dark side of the Internet requires dedicated software. We covered the theory behind a tool made for this purpose: The Tor project. An open-source initiative designed to help users browse the Internet anonymously using a technique known as “The Onion Routing”.
The theory is good, but sometimes not enough. In this post, we will teach you everything you need to start exploring the dark side of the Internet with the help of Tor.
Quick Recap
Previously, we embarked on a journey into the unknown. We explored a world beyond our everyday activities on the Internet, understanding the different ways we can access information. Furthermore, we saw the multiple layers of the Internet: the Surface Web, the Deep Web, and the Dark Web.
Accessing the Surface Web and the Deep Web is an activity we do every day: to get information from the Surface Web, we need a browser and search for some data on a search engine. Similarly, to access the Deep Web, we don’t need special software but a regular browser and the necessary credentials to access the information.
As for the Dark Web, we have two options to browse the dark side of the Internet using the Tor project. Let’s start with the easiest one.
The Tor Browser
The Tor browser is like any other: it allows you to search for any website on the Surface Web (or the Deep Web if you have the correct credentials). The main difference is that, in addition, we can connect to the Tor network out of the box. Thus, the Tor browser is the easiest way to start surfing the Dark Web.
The Tor project team built the browser from a modified version of Firefox ESR (Extended Support Release). First, you need to download it and follow the installation instructions from their website: https://www.torproject.org/download/
Onion services (formerly known as “hidden services”) are services (like websites) that are only accessible through the Tor network.
Before continuing, let’s see what happens if we try to access one of these sites in a “regular” browser. For this example, we’ll use Google Chrome. Feel free to use your preferred browser.
When we visit the following URL: https://www.bbcnewsd73hkzno2ini43t4gblxvycyac5aw4gnv7t2rccijh7745uqd.onion on Google Chrome, we’ll notice that we can’t reach the page.
The URL should’ve taken us to the BBC News Onion service. Instead, the page is not reachable. If you tried another browser like Firefox, Safari, or Edge, you may have gotten the same result.
That is a hint that we’re not using the Tor network. Additionally, the Tor project has a website we can use to check whether we have a connection to the Tor network.
Once we have the browser correctly installed, when we open the application, we will be asked whether we want to join the Tor network or surface the Web without it.
In this example, we’ll click the purple Connect button to establish the Tor network connection.
After establishing the connection, we can check the onion service once more. We’ll notice that the Tor browser displays the page without problems.
The method above is straightforward. After following a few steps, we were ready to navigate through the Dark Web. This approach is perfect for you if you want to explore the dark side of the Internet. However, the Tor browser is insufficient when building more sophisticated tools that leverage the Tor project.
For example, what if you want to create a program that gets access to the Dark Web and processes some news from the Onion service we saw earlier? The following section is perfect for this kind of need.
Connecting Programmatically to the Tor network
Next, we will programmatically connect to the Tor network using Go as our programming language. To do this, make sure you meet the following requirements:
-
Make sure you have Go installed in your system. You can download it from the official Go website. Ensure you have Go installed. You can verify by running:
go version -
Install the Tor software from their website.
- If you use mac, you can install it using Homebrew as follows:
$ brew install tor
Once installed, start the Tor service. You can do that using the command line:
$ tor
Ensure Tor is running correctly by checking the logs for any errors.
- Additionally, we will use Bine. A package to bridge our Go application with the Tor network:
$ go get github.com/cretz/bine
For the following examples, we will connect to the Tor network both as a client and as a server to host Onion services.
Client Example
You can use a Tor instance to connect to the Tor network in two different ways. The following example requires you to initiate the Tor instance on a new terminal tab as the first step. If you correctly installed Tor, open a terminal window and use the tor command to start it.
Once you have Tor running, you can use the code below:
package main
import (
"bytes"
"fmt"
"golang.org/x/net/html"
"log"
"net/http"
"strings"
)
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
func run() error {
t, err := tor.Start(nil, nil)
if err != nil {
return err
}
defer t.Close()
dialCtx, dialCancel := context.WithTimeout(context.Background(), time.Minute)
defer dialCancel()
dialer, err := t.Dialer(dialCtx, nil)
if err != nil {
return err
}
httpClient := &http.Client{Transport: &http.Transport{DialContext: dialer.DialContext}}
resp, err := httpClient.Get("https://check.torproject.org")
if err != nil {
return err
}
defer resp.Body.Close()
parsed, err := html.Parse(resp.Body)
if err != nil {
return err
}
fmt.Printf("Title: %v\n", getTitle(parsed))
return nil
}
func getTitle(n *html.Node) string {
if n.Type == html.ElementNode && n.Data == "title" {
var title bytes.Buffer
if err := html.Render(&title, n.FirstChild); err != nil {
panic(err)
}
return strings.TrimSpace(title.String())
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
if title := getTitle(c); title != "" {
return title
}
}
return ""
}
If we run the program above, we can successfully connect to the Tor network and access any .onion site. Let’s break down what happens:
t, err := tor.Start(nil, nil)
if err != nil {
return err
}
defer t.Close()
To begin with, we connect to The Tor instance using Bine, the library we downloaded previously. Bine works as a bridge between our application and the Tor instance.
dialCtx, dialCancel := context.WithTimeout(context.Background(), time.Minute)
defer dialCancel()
dialer, err := t.Dialer(dialCtx, nil)
if err != nil {
return err
}
httpClient := &http.Client{Transport: &http.Transport{DialContext: dialer.DialContext}}
Then, we use the Tor instance to set up a client to make requests through the Tor network, providing anonymity and privacy for the connections:
-
We create a context with timeout: This ensures that if establishing the Tor dialer takes too long (more than a minute), it will cancel the process to avoid hanging indefinitely.
-
Use the context to create a Tor dialer that handles all outgoing connections via Tor.
-
Configure the HTTP client to use the custom transport that routes all traffic through the Tor dialer.
Under the hood, Tor uses SOCKS5, a protocol for routing network traffic through a proxy server. The protocol serves to bypass firewalls or access servers in different geographic locations. In addition, Tor uses its protocol and other technologies to provide a higher level of security and anonymity.
resp, err := httpClient.Get("https://check.torproject.org")
if err != nil {
return err
}
defer resp.Body.Close()
parsed, err := html.Parse(resp.Body)
if err != nil {
return err
}
fmt.Printf("Title: %v\n", getTitle(parsed))
return nil
Finally, we use the client to make an HTTP request to the check.torproject.org website. Then, we get the response and use the html.Parse method to parse the content of the body. Finally, we use a handy function to traverse the HTML tree, get the title, and print it in the terminal. After we run the program, we can see that we have a successful connection to the Tor network:
$ go run main.go
Starting Tor and fetching title of https://check.torproject.org, please wait a few seconds...
Title: Congratulations. This browser is configured to use Tor
As an exercise, you can try accessing the same URL we used before: https://www.bbcnewsd73hkzno2ini43t4gblxvycyac5aw4gnv7t2rccijh7745uqd.onion
The example above is great when we want to read or consume data from the Dark Web pages. However, what if you’d like to launch your Onion services? Let’s check how that works.
Server Example
For the server example, let’s see an alternative way to connect to the Tor instance. Here, instead of manually starting the Tor instance, we provide the executable path.
package main
import (
"bytes"
"fmt"
"golang.org/x/net/html"
"log"
"net/http"
"strings"
)
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
func run() error {
t, err := tor.Start(context.TODO(), &tor.StartConf{ExePath: "/opt/homebrew/bin/tor", TempDataDirBase: "tor"})
if err != nil {
return err
}
defer t.Close()
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, Dark World!"))
})
// Wait at most a few minutes to publish the service
listenCtx, listenCancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer listenCancel()
// Create an onion service to listen on 8080 but show as 80
onion, err := t.Listen(listenCtx, &tor.ListenConf{LocalPort: 8080, RemotePorts: []int{80}, Version3: true})
if err != nil {
return err
}
defer onion.Close()
// Serve on HTTP
fmt.Printf("Open Tor browser and navigate to http://%v.onion\n", onion.ID)
return http.Serve(onion, nil)
}
The code above sets up and runs an HTTP server accessible through the Tor network by performing the following steps:
t, err := tor.Start(context.TODO(), &tor.StartConf{ExePath: "/opt/homebrew/bin/tor", TempDataDirBase: "tor"})
if err != nil {
return err
}
defer t.Close()
Again, our first step is starting the Tor instance:
-
t.Startinitializes and starts a new Tor process. This time, we pass acontext.TODO()(a placeholder context): indicates that you might want to replace it with a more specific context if needed.context.TODO()is equivalent to passing nil as the argument. -
&tor.StartConf{ExePath: "/opt/homebrew/bin/tor", TempDataDirBase: "tor"}specifies the configuration for starting Tor. -
ExePathis the path to the Tor executable. You might need to use a different path depending on where is Tor on your machine. -
TempDataDirBasespecifies a base directory for temporary data. This directory is used only by Tor internally.
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, Dark World!"))
})
// Wait at most a few minutes to publish the service
listenCtx, listenCancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer listenCancel()
Then, we set up an HTTP Handler that responds with Hello, Dark World! to incoming HTTP requests. We also create a context with a timeout that cancels after 3 minutes to manage the creation of the Onion service.
// Create an onion service to listen on 8080 but show as 80
onion, err := t.Listen(listenCtx, &tor.ListenConf{LocalPort: 8080, RemotePorts: []int{80}, Version3: true})
if err != nil {
return err
}
defer onion.Close()
// Serve on HTTP
fmt.Printf("Open Tor browser and navigate to http://%v.onion\n", onion.ID)
return http.Serve(onion, nil)
Finally, we create and start serving the Onion service by using the Tor instance:
-
The
Listenmethod configures and starts an onion service that listens on port8080locally and exposes port80remotely. -
The
http.Servemethod starts the HTTP server on the onion service and prints the.onionaddress for access.
By running this program, we have a Tor-hidden service that serves an HTTP endpoint accessible via Tor.
As we can see, compared to the Tor browser, connecting programmatically to the Tor network requires more effort, but this will open a whole new world of opportunities for building technological solutions for our everyday lives.
Conclusion
Venturing into the Dark Web might initially seem intimidating, but with the right tools and knowledge, all the dots connect.
In this post, we’ve navigated through using the Tor Browser for simple access and explored advanced techniques to establish a connection to the Tor network programmatic connections using Go. Whether you’re investigating or building advanced tools, always contemplate the ethical implications and legal boundaries of your activities.
Following the steps outlined in this post should give you a solid foundation for accessing the Dark Web and integrating Tor into your projects. The journey doesn’t end here; continue exploring, learning, and innovating while committing to the responsible and ethical use of the powerful technologies at your disposal.
See you in the next one!
