Deep web
Download as PPTX, PDF2 likes1,001 views
The document discusses the deep web, which contains vast amounts of information not indexed by standard search engines, making it significantly larger than the surface web. It covers historical terminology related to the deep web, its classification, and the technologies used to navigate it, like web crawlers and the Tor network for anonymity. Additionally, it mentions the existence of illegal content and activities within the deep web, as well as the use of Bitcoin for transactions in such environments.
Deep web
- 1. Present by: Animesh Pramanik Prasanjit Mondal (IT 3rd Year) Jalpaiguri Government Engineering College(Autonomas)
- 2. A large portion of data available on the web is present in the so called deep web.. World Wide Web content that is not part of the Surface Web and is indexed by search engines. It is called the Deep Web, Invisible Web or Hidden Web. Largest growing category of new information on the internet. For example, web pages regarding private user accounts are in the deep web (Private Info). The Deep Web is the majority of online content, estimated to be 400-550 times larger than the surface web.
- 3. Early Days: static html pages, crawlers can easily reach In mid-90’s: Introduction of dynamic pages, that are generated as a result of a query. Jill Ellsworth used the term “Invisible Web” in 1994 to refer to website that were not registered with any search engine. Another early use of the term invisible Web by Bruce Mount and Matthew B.Koll of personal Library software in 1996. The first use of the specific term “Deep Web”, now generally accepted, occurred in the aforementioned 2001 Bergman study.
- 4. “… when you can measure what you are speaking about, and express it in numbers, you know something about it…” – Lord Kelvin First Attempt: Bergman (2000 ) Size of surface web is around 19 TB Size of Deep Web is around 7500 TB Deep Web is nearly 400 times larger than the Surface Web.
- 5. In 2004 Mites classified the deep web more accurately Most of the html forms are found either on the fist hop or 2nd hop from the home page
- 6. Unstructured: Data objects as unstructured media (text, images, audio, video) Structured: data objects as structured “relational” records with attribute-value pairs.
- 8. Task Specific Database LVS Manager Label Value Set (LVS) Form Analyzer Parser Form Processor URL List Crawl Manager Response Analyzer WWW Data Sources … Response Feedback Form Submission
- 10. Surface Web Deep Web Entries are statically generated Linked Content (web crawled) Readily accessible through any browser or search engine unlike the Deep Web, which requires special search engines, browsers, and proxies to access. Entries are dynamically generated (submitted to a query or accessed via form). Unlinked Content Contextual Web Private Web Scripted Content Non-HTML content Limited Access Content (anti-robot protocols like CAPTCHA)
- 11. Advantages Of Invisible Web Content Specialized content focus – large amounts of information focused on an exact subject Contains information that might not be available on the visible web Allows a user to find a precise answer to a specific question Allows a user to find webpages from a specific date or time
- 12. Search Engine construct a database of the web by using programs called spiders or web crawlers that begin with a list of known Web pages. The spider gets a copy of each page and indexes it, storing useful information that will let the page be quickly retrieved again latter. Any hyperlinks to new pages are added to the list of pages to be crawled. Eventually all reachable pages are indexed, unless the spider runs out of time or disk space. The collection of reachable pages defines the Surfs Web .
- 18. The web that the vast majority of internet users are accustomed to. Accessible in any nation that does not block internet access, even places like China and Egypt. Social media sites like Facebook, informational websites like Wikipedia, general websites, etc.
- 19. The layer of the Surface Web that is blocked in some nations. Some other information is only accessible through illegal means. Google locked results Recently web crawled old content Pirated Media
- 20. Requires a proxy or two (namely Tor) to access. Contains most of the archived web pages of the 1990s Web that did not renew their domain names and such. Government/Business/Collegiate Research. Hackers/Script Kiddies/Virus Information. Illegal and Obscene Content (CP, Gore, Suicides, etc.)
- 21. Like the Regular Deep Web, but harder to get into and more illegal content. Advanced covert government research. Most of the internet black market (run on bitcoins) Human/Arms/Drug/Rare Animal Trafficking. Assassination networks , bounty hunters, illegal game hunting, line of blood locations, etc. More banned obscene content like CP, Gore, etc.
- 22. Lowest known level of the Deep Web. Named after the Spanish Technician who created it. Extremely difficult to access, users say it is the safest part of the internet due to how private it is. Julian Assange and other top-level Wikileaks members are believed to have access.
- 23. You may wonder how any money-related transition can happen when sellers and buyers can’t identify each other. That’s where Bitcoin comes in. Bitcoin, it’s basically an encrypted digital currency. Like regular cash, Bitcoin is good for all transactions of all kinds and notably, it also allows for anonymity; no one can trace a purchase, illegal or otherwise. When paired properly with TOR, it’s perhaps the closest thing to a foolproof way to buy and sell on the web.
- 24. Tor is the software that installs into your browser and sets up the specific connections you need to access dark Web Site. Critically it is free software enabling online anonymity and censorship resistance. Onion routing refers to the process of removing encryption layer from internet communication, similar to peeling back the layer of an onion. Using Tor makes it more difficult to trace Internet Activity, including “visiting to web sites, online posts, instant message, and other communication forms”, back to the user.
- 25. It is intended to protect the personal privacy of users, as well their freedom and ability internet activists from being monitored. In send of seeing domains that end in .com, .in or .org, these hidden sits end in .onion. The FBI eventually captured Ross Ulbricht, who operated Silk Road, but copycat sites like Black Market Reloaded are still readily available. Tor is the result of research done by the U.S. Naval Research Laborite, which created Tor for political dissidents and whistleblowers, allowing them to communicate without fear of reprisal.