Dive into the Deep Web (but watch where you swim)

Deep Web Image

If you ever watched the rain fill a hole in the ground, then you can understand where the term Deep Web comes from.

For the past 10,000 days – the approximate age of the World Wide Web – we’ve poured gallon after gallon of content into that vast networking structure known as the Internet and watched as that content seeped into every crevasse of our lives. And the number of sources is as vast as the structure itself; none of us truly knows where all that content originates.

Now, imagine that, instead of overflowing, the hole gets deeper and deeper to contain the content pouring into it. You can see across the surface and maybe a little below it. But other content submerges to where you need special tools for access.

Search engines such as Google and Yahoo! and web browsers such as Firefox merely skim this surface, collecting indexed information from its source. These kinds of tools probe only about 5 percent to 10 percent of the Web’s content.

Deep-Web diving, on the other hand, reveals the immense amount of information not indexed by standard search engines. Much of it is exchanged through peer-to-peer networks and resides on databases, unregistered websites, query-sensitive dynamic pages, limited sites, non-HTML sites, broken or hidden web links and backlinks, scripted content, and web archives, among other sources.

The list of useful deep-diving tools is long, but among the most common tools are Freenet, IceRocket, I2P, SurfWax, the WWW Virtual Library, a series of search applications provided by Deep Web Technologies, and the Tails operating system. There are also customized tools targeting specific caverns nestled in the Deep Web.

A word of warning, however: The deeper you go, the darker the Web gets. This is why in recent years the terms “deep” and “dark” have become conflated regarding the Web. At Deep Web’s bottom layer, there be dragons who dabble in questionable or outright illegal behavior. Using Tor, a free browser designed to protect the user’s anonymity, deep divers can peer into portions of this darker area.

Granted, not everyone at this depth wears a black hat. Good guys dwell down there, too, such as journalists, law enforcement, the military, and whistleblowers. But like anywhere else, trouble can be found if you go looking for it. So, exercise the same caution swimming in the Deep Web as you would in deep water. Keep a lifeline handy like this one (accessible through Tor) and enjoy the voyage.