You've probably heard the term dark web, deep web, dark web, or other similar names in passing before, the common - but rarely talked about - topic of the 'deep web. A web under the web, filled with petabytes of data and information that are beyond the reach of the standard search bar of Google, Bing, or Yahoo.
But what exactly is the Deep web, and what purpose does it serve in the larger search community as a whole? Read on for our guide to find out everything you need to know about the deep web, including what it means, where you live, and how you can use it to your advantage.
The surface Internet, which is the Internet that we know and uses daily, such as “Google”, “Microsoft”, Facebook, Amazon, and other millions of other sites, where the number in 2020 was estimated to reach 550 million sites registered and available for a visit via the surface Internet, but the shocking thing to you is that this huge number of The available sites are only 6% of the total network content, and the rest is found in the deep and dark Internet, whose hidden sites are estimated at tens of billions and contain various prohibitions and prohibitions and the rest of the illegal matters. What is the deep Internet and what is the difference between it and the dark Internet?
The deep internet or the deep web
If the visual Internet is all that search engines can reach, then the deep Internet is all that search engines cannot reach, that is, it is just a deep area.
The deep areas of the web are not secret, on the contrary, they can't be indexed, no secrets here actually, it just depends on how the search engines work. Search engines rely on links, that is, they provide you with a link to visit a site that contains the keywords you wrote in the first place, and then you click on the links that you think will lead you to your destination. But it won't help you further than this!
Moreover, some pages can be blocked from indexing, that is, although they have a link, you will not find them by any means of search, but you have to go to them yourself. This means that you have to visit those specific places directly instead of searching for them, and this depends on the privacy of the site, for example. or the features it provides to its users.
Like your posts on Facebook, which have the privacy of “only me” and in fact, not even your friends can see them, there is one way to access these posts and that is to log into your account, then your page, and so on.. the same applies to search engines if you specify certain privacy for the article or I left it to be archived in search engines.
Let's take an example of this (what search engines can't access) the famous Amazon site. You can search traditional search engines to get to this site in particular, or even to find a specific product within this site by providing its name before, but try, for example, to search on Google for iPhone cases ranging in price between 20-50 dollars and added four days ago with shipping Complimentary. Then you'll find links to everything but a result with that exactness and specificity you asked for! And why? Because what you requested does not have a direct link and the search engine cannot recognize it. This is a “deep” area that a search engine cannot reach, and the only thing that can help you here is to use the search bar within the site itself because the site necessarily indexes and categorizes the products and items inside it, which enables him, and only him, to search for something that no one knows exists yet ...something that does not exist at all unless the researcher wants it to exist!
So we see that the deep internet doesn't necessarily mean secret content, it's as secret as the grocery store down your street! It's secret to most people because they don't live in the same street or they don't know how delicious those black olives they sell are! This is not confidential information, it is only “deep” and invisible to passersby from afar, as all they know is that this is a residential neighborhood named so with a simple description!
All this means that search engines do not use the search bar on sites, but rather the links provided by each site and below that remain as part of the content of the deep Internet. So this could be disappointing! So it is not as scary or secretive as the media is promoting; It is only a certain type of content that requires a special search facility. But it's about to change if we go a little deeper...
What is the dark web or the dark web?
“Dark web” is an umbrella term for a part of the Internet that is not indexed by search engines; Contains sites that do not appear in Google searches. What you need is the Tor browser to allow you to access .onion websites, which are only available on the Tor network.
The dark web is an area of the internet that can only be accessed through a specific browser program, such as Tor or I2P. It is a network of anonymity where users' identities and locations are protected by encryption technology that routes user data through numerous servers around the world - making it extremely difficult to track users.
The anonymity of the dark web makes it an attractive technology for illegal purposes. Unfortunately, getting visibility into criminal websites is difficult: it requires expert knowledge, access to closed sources, and technology capable of monitoring these sources for misuse of your data.
However, let's first start to dispel some of the misconceptions about the dark web.
Assumption 1: The dark web is synonymous with the criminal internet. While the dark web is home to a lot of crime, it also hosts many legitimate companies like the New York Times and Facebook that provide Tor-based services, as well as generally harmless content. The dark web is not synonymous with cybercrime.
Second assumption: the dark web is the same as the deep web. To be clear, the deep web is broadly defined as anything that hasn't been indexed by traditional search engines. Unsurprisingly, the Deep Web is also home to criminality - but so is the Clear Web. The dark web does not have a monopoly on cybercrime.
Simply because it can't be accessed by the traditional search engine, it doesn't necessarily mean that the deep web is interesting. Most of the data on the deep web is mundane or "ordinary"; For example, email or Facebook accounts may fall under this definition because they require registration to view the content. While some deep and dark websites are valuable sources, you need to know what you're looking for, or else it's easy to waste time and resources.
If you've ever paid attention to online marketplaces for illegal goods like the now-defunct Silk Road or the FBI's investigations into crimes in cyberspace, you've likely heard the term "dark web." Curious about what that means? You have come to the right place.
The dark web is sometimes called the Onionland due to its content that can only be accessed using services like Tor. The rest of the Internet is simply referred to as the Clean Web since it is generally unencrypted.
How does the dark web work?
The dark web works just like the regular internet: it uses the same TCP/IP framework to transmit HTTP and FTP traffic within and between networks, over the same phone, cable, or FiOS lines that carry regular internet traffic. The content on the dark web is made up of HTML web pages and their assets, just like on the rest of the web. In fact, under the hood, the dark web is the same as the regular web, with two important exceptions that also distinguish the dark web from the deep web.
First: the dark web is not indexed by search engines. Second, content on the dark web cannot be accessed using regular web browsers alone; Additional software is required to make the networks talk to each other.
This is because the content on the dark web is hosted on overlay networks that are physically connected to the internet but cannot be accessed by web crawlers. The reason for this relative inaccessibility is because the dark web uses an entire network addressing system, but is fundamentally different from the web addresses that most of us know and use. Browsers like Chrome and Firefox are programmed to access website files using a DNS index, which converts the unique address of a file on its server that is unique to a text string that you can type into the address bar. Sites indexed by the DNS record can be accessed via top-level domains such as .com and .org, among others. After ICANN opened up the suffix system for other text strings, we started seeing web addresses that looked like home. CERN and bit.ly - but you can still type those addresses into the address bar and access a website because they're in the official DNS record.
Tor Browser
The blocked content can still be accessed this way, but you need the right software. It's a bit like a Wi-Fi network that doesn't broadcast its SSID: you can only access it if you already know exactly how to find it. Some of the content that can only be accessed through Tor is hosted in a pseudo-top-level domain .onion, which means that in the right program, you can type foobar. onion and access the dark Foobar site.
Such programs, including the Tor Browser Bundle, can bridge the differences in network behavior between the dark web and the clean web. But this only works when you are using a compatible browser and have the correct encryption. Tor, Freenet, and I2P are the most commonly cited examples of software capable of accessing the dark web. Typing the .onion address into the Chrome address bar won't get you anywhere. Moreover, many websites are created.
How Tor Browser Works
There is also a difference in the path that web traffic takes on the clean or surface web versus the dark web. Tor is valuable because it sends your web traffic through several different network nodes, hiding its origin and destination. There is a significant overlap between VPNs and the dark web; Both services use encryption and multiple network nodes to anonymize traffic. But VPNs deal with clear websites that participate in the DNS, while dark web browsers deal with domains that ICANN does not recognize.
dark web search engine
Dark web search engines exist, but even the best are challenged to keep up with the ever-changing landscape. This experience reminds us of searching the web in the late 1990s. Even one of the best search engines, called Grams, returns results that are repetitive and often irrelevant to the query. Link lists like The Hidden Wiki are another option, but even pointers also return a frustrating number of timed-out connections and 404 errors.
dark web sites
Dark web sites look pretty much like any other site, but there are important differences. One is the naming structure. Instead of ending in .com or .co, dark web sites end in .onion. This is a "private use top-level domain suffix that identifies an anonymous hidden service that can be accessed over the Tor network," according to Wikipedia. Browsers with the appropriate proxy server can access these sites, but others cannot.
Dark web sites also use a mixed naming structure that creates URLs that are often difficult to remember. For example, a popular trading site called Dream Market goes to the incomprehensible address "eajwlvm3z2lacca76.onion."
Many dark websites are created by scammers, who are constantly on the move to avoid the wrath of their victims. Even commercial websites that may have been around for a year or more can suddenly disappear if their owners decide to cash in and flee with the escrow money they hold on behalf of clients.
Law enforcement officials are getting better at finding and prosecuting owners of sites that sell illegal goods and services. In the summer of 2017, a team of cyber cops from three countries successfully shut down AlphaBay, the largest source of contraband on the dark web, sparking panic across the network. But many merchants simply migrated to other places.
The anonymous nature of the Tor network makes it particularly vulnerable to DDoS, said Patrick Ticket, director of security and architecture at Keeper Security and the company's resident expert on the subject. "Sites are constantly changing addresses to avoid DDoS, which makes for a very dynamic environment," he said. As a result, “the quality of research varies widely, and much of the material is outdated.”
What are the uses of the dark web?
The structure of the dark web makes it anonymous, which means that first of all, it is used for anonymous communication and web browsing. This accounts for the vast majority of network traffic through Tor. Why seek anonymity? Read and write about things that might get you in trouble, such as political opposition or whistleblowing. The same technology that enables Tor to penetrate tunnels from behind the Great Firewall of China, and the US government contributes to the development of such programs.
Anonymity also highlights those who wish to do illegal things. A 2014 study found that among the different types of sites on the dark web, there are more markets devoted to drugs and weapons than any other type of dark web, including forums, bitcoin laundering, hacking, fraud, whistleblowing, and even old porn.
To paraphrase Jim Jefferies, if you want to kill someone, you can't just go up to Pier 31 and shout "Guns, who wants to sell me some!?" But with a website like evil eBay listing guns and other contraband for sale, suddenly you don't have to know someone with "black market connections." You just have to be able to install some software.
Over 110,000 users and 5,400 sellers on Wall Street
Tor hidden services are the other thing the dark web does, and they are what gives the dark web its shady reputation. Hidden services refer to dark sites where both the host and the visitor are anonymous to each other. This technology enables dark websites hosting illegal content to survive. Hidden services make up only 1.5% of the Tor network. But the vast majority of the resources requested via Tor's hidden services — entirely 80% of that traffic — were requests from child abuse sites. Traffic from the dark web primarily flowed between bot networks and hidden control servers. More details about Tor traffic patterns and the total amount of bandwidth used for illegal activities are available in a blog post by the Tor Project.
The Dark Web is notorious for being an elusive area for both buyers and sellers. Law enforcement is cutting back on the nominal anonymity offered by programs like Tor, and anything interesting on the dark web is likely to be a scam as much as it is an attraction. Between social engineering and software vulnerabilities, it is an area that can be better accessed with the use of some reliable anti-malware software.
For a long time, the Silk Road was the biggest game in the dark web trade. It has allowed users to sell a very large number of illegal things and has inspired several similarly designed counterfeit markets. Transactions were made there with bitcoin and other virtual currencies, and then the goods were shipped by mail. But a major bankruptcy and the ensuing court case put several Silk Road officials in jail. The media spotlight affected the relative obscurity of the Silk Road, reducing its value as a black market.
While Uncle Sam contributes to the development of Tor and similar anonymizing resources, the government has also been known to take a more proprietary approach, considering that the Dark Web falls within US jurisdiction when site hosting is in question. The FBI paid Carnegie Mellon to crack and hack Tor in a criminal case. They even waded through the mud and performed a massive stinging operation on Playpen, a child porn site on the dark web - by taking control of the site and operating for weeks as a trap to lure and capture its users.
A man points to the web page of an online store selling machine guns on the dark web in Germany on November 12, 2016. Silas Stein/picture alliance via Getty Images
A man points to the web page of an online store selling machine guns on the dark web in Germany on November 12, 2016. Silas Stein/picture alliance via Getty Images
The dark web is an excellent example of how difficult it can be to prevent criminals from using anonymizing services designed to protect honest opponents. Tor's anonymizing functionality is very important for people who rely on it to discuss sensitive topics without fear of reprisal. The debate about how much light should be shed on the Dark Web is a topic of ongoing debate. How much illegal activity should be allowed to preserve the positive benefits of Tor, and is there a way to detect pedophiles and other illegal activities without compromising the security that makes the dark web work?
The bright side of the dark web
The Dark Web has had some seriously bad press. Everything from the spread of child pornography is blamed for facilitating online fraud, dealing hard drugs, hacking email boxes, selling malware, supporting ISIS, and even allowing people to hire contract killers. But what if there was a brighter side to the phenomenon that we learned to see as "dark" and "shadowed"? While it would be foolish to ignore very real abuses on the dark web, there is a case to be defended, from freeing opponents to share information, to protecting whistleblowers from persecution. Here are three reasons to give it a second look.
Exhibitions' best friend
The modern world is full of regimes thirsty for information about their citizens. They may vary in degrees of repression and brutality, but governments from Beijing to Washington, DC, are willing and able to monitor information and put opponents behind bars if the facts they hold are inappropriate.
The Dark Web complicates the mission of the secret police, allowing whistleblowers to leave bits of information about anything from illegal diamond mining to nuclear arsenals. For example, in 2013, the Guardian revealed that the National Security Agency (NSA) was monitoring the phone calls of US citizens, whether they were suspected of a crime or not. Edward Snowden turned to the Dark Web when he leaked several pieces of information that helped expose NSA surveillance. The dark web has proven so useful to whistleblowers that the CIA itself launched a Tor-based site to receive leaks from its sources. So, while the Dark Web has been butchered in some corners of the organization, at least one organization is seeing its brighter side.
A place to escape repressive censorship
The Dark Web isn't just a vital protective shield for whistleblowers. It is also a way to evade censorship and ensure that democratic debate continues, safe from the prying eyes of the state. The revelations by the National Security Agency showed the extent of surveillance. But censorship is no less dangerous, albeit less dangerous in the United States. Take China, for example, where the dark web has become an essential tool for hiding communications from the country. With 200 million cameras in the Skynet system, a "Great Firewall" monitoring data leaking from the country, and rampant use of apps by the government to track phones, dissidents would be fools not to use Tor and the Dark Web.
But this is not just a Chinese problem. Nation-states all over the world know that the Internet is a dangerous force. From Uganda to the United Arab Emirates, despots have the power to block standard communications, force websites to close, and monitor users — making the lives of despots much easier. As Cindy Cohn of the Electronic Frontier Foundation (EFF) said, "We see Tor usage rise whenever a dictatorship takes over or a coup takes place. Tibetans, UAE, Tunisia, Egypt. The list goes on and on."
Tor and the dark web hold hope. When Venezuela tightened censorship on the web, Tor forums filled with angry citizens, fueling resistance. The Russians have used Tor routinely for years, and the same goes for the Saudis and Iranians. It even helped trigger the "Arab Spring" of 2010-2011. Tor has become so important to dissidents that Facebook created a special Tor portal for protesters to use. Without it, organizing resistance movements around the world would be much more difficult.
Free Press Support
When journalists investigate the powerful, they inevitably encounter opposition. All too often, journalists find their sources become uncooperative, particularly when they investigate terrorist states, companies, or groups. As a result, dark web tools such as Secure Drop have become widely popular among journalists to securely receive information from sources. Researchers also use Tor to connect to their sources. In many cases, there is no other way to communicate without fear of disruption or discovery. ProPublica has been a pioneer in this type of journalism, creating a Tor site dedicated to research resources and projects. With these safeguards in place, it has provided an effective tool to challenge the powerful - something only Tor encryption can provide.
As with all technology, the dark web is just as bad - or good - or as we are harnessing it. Overcoming censorship, protecting journalistic sources, and helping whistleblowers are all huge benefits to the dark web. With killers and scammers in decline, the dark web could become freedom's most important ally.