This blog sheds some light on the term OSINT, its types, actors interested in OSINT gathering, exploration, and what benefits OSINT provides security researchers, analysts, and professionals today.
OSINT is an acronym for Open-Source Intelligence and forms one of the key concepts in building a robust cybersecurity system. OSINT is the practice of collecting information from already published sources or public sources available on the Internet. The OSINT operation process performed by IT Operatives, malicious actors, or sanctioned intelligence operatives uses advanced search techniques that are publicly available to gather information. Open-source in OSINT does not refer to the open-source software movement, but rather, points to the public nature of the data, which is freely available on the Internet.
Collating data helps in many ways, such as building a robust cybersecurity system by reducing your attack surface and securing information available publicly. It also helps you to gain a competitive advantage and get a jump-start on your competitors.
🔹Simple OSINT examples include:🔹
🔹Asking questions on any search engine.
🔹Research public forums on the latest mobile technologies.
🔹Watch a YouTube video on how to make a pick for lockpicking.
🔹Importance of OSINT🔹
OSINT, in general, helps an organization keep tabs on publicly identifiable information. It also helps in reducing the potential attack surface and thus prevents breaches and leaks. For example, the following tasks are done with the help of Osint techniques and methodologies:
🔹Discovering and Locating the Assets Outside the Perimeter, or Public:🔹
OSINT helps the IT and the cybersecurity teams discover and locate public-facing assets. Through OSINT, the information available in each asset can then be mapped and assessed for sensitive or critical information that can be exposed or exploited. In general, the OSINT tools help in mapping and recording data of the public assets of the company that is publicly available and accessible.
🔹Finding Relevant Data and Information Outside the Organization:🔹
The OSINT tools help find relevant data outside the organization, such as domains or ports outside the organization’s internal network perimeter. This function is particularly helpful for an organization that has recently merged or acquired another organization as it helps to find relevant information available outside of the organization just acquired.
🔹Necessary Measures with Collated Data:🔹
The data collected can be massive and also not in order. OSINT tools convert the data into meaningful information that can be used as actionable intelligence. OSINT tools also help to piece the data together and deals with sensitive data and their problems on a priority basis.
🔹Define OSINT Techniques🔹
🔹What are you looking for?
🔹What is your main research objective?
🔹Who is your main target?
🔹What tools or mechanisms will you undertake to conduct the research?
While there are a lot of OSINT methodologies and mechanisms available, not all of them will help to achieve the target. First, you need to define the scope of the search and identify and prioritize security gaps and flaws.
OSINT techniques can be divided into two major categories, namely Active OSINT, and Passive OSINT.
🔹Active OSINT: Includes port and system scanning and direct contact with the target. the results are more reliable and dependable, along with a high risk of detection.
🔹Passive OSINT: In this category, contact is established with the help of third-party services. Since it includes a third-party, the search results may not be reliable and may include many false positives and negatives; therefore, the risk of detection is quite low in this category.
🔹Risks Involved with OSINT Tools🔹
🔹Getting detected: This is the most common risk involved as performing an OSINT investigation may give your information away as the one who was searching for the data.
🔹Losing access to information: Getting detected may result in you losing access to the information as it may lead to securing publicly identifiable information or hiding the trails/covering/covert tracks.
🔹Top OSINT Tools🔹
🔹You become the victim: If your cover is blown, you can risk becoming a target of an investigation, or even worse, spying.
🔹Content Filtering: OSINT performs an exponential search and collates a humongous amount of data. if the data is not pieced together, or in order correctly, the data collected is then useless and doesn’t result in any meaningful action.
🔹Counter the Challenges in Performing OSINT:🔹
1. BuiltWith: As the name suggests, BuiltWith lets you decode, or find out, what the websites are built with or made up of. It enables the user to identify different tech stacks and platforms that power the websites. For instance, BuiltWith can identify whether the website is built using Joomla, WordPress, or Drupal as its CMS. It also identifies and generates a list of JavaScript/CSS libraries, website plugins, website frameworks, and server information. BuiltWith can be used as preliminary research or an observation tool for websites.
2. Maltego: Maltego is primarily used for uncovering relationships among domains and publicly accessible information. It also helps in chartering the humongous data into readable and easy-to-understand charts and graphs, which helps convert the raw data into meaningful usable data. Maltego comes with 58 data integrations from over 35 data partners that allow users to choose four [4] different layouts to recognize patterns in the data they’ve uncovered and piece complex data together.
3. Mitaka: Mitaka is available as a Chrome extension and as a Firefox browser addon that helps in searching IP addresses, URLs, domains, hashes, and wallet addresses across six [6] dozen search engines. It also helps the cyber security team recognize and detect various Indicators of Compromise [IOCs] from your web browser and helps mitigate threats and risks. Additionally, as they are extensions, online databases can be quickly queried with just one click.
4. Spyse: Spyse is considered the complete internet assets registry and is used to collect the data from servers, websites, and peripheral connected devices which are also publicly available. This is also used as a reconnaissance tool that conducts data analysis to detect any security vulnerabilities of unmanaged assets and also helps in securing exposed credentials.
5. Spiderfoot: Spiderfoot is a free OSINT reconnaissance tool that integrates with different multiple data sources and automates the collection of OSINT. Spiderfoot gathers and analyzes data regarding domains, IP addresses, CIDR ranges, phone numbers, usernames, and other sensitive data. Providing an intuitive web-based GUI, Spiderfoot contains both a command-line interface and an embedded web server, making it ideal for Red Team reconnaissance activities. In general, Spiderfoot helps to discover more information about your target or identify what your organization may be inadvertently exposing to the public.
6. OSINT Framework: The OSINT Framework doesn’t run on servers, but is a web-based interface that is useful in gaining valuable information and data by querying free search engines, resources, and tools and helps you sniff out the data you need by breaking down the different topics of interest. The querying of websites for data extraction is free, while some require registration and/or have paid versions that help you construct an advanced google search and provides help in collecting in-depth data.
7. Creepy: Creepy is a tool written in Python that helps in collecting geolocation data of any individuals through a query raised through social networking platforms and hosting services. Creepy enables the user to present or plot the data collected on a map. It also allows the users to download or filter the data.
8. Recon-ng: Recon-ng is a tool also written in Python distributed via Kali Linux arsenal of pentesting tools that comes fully stocked in the Kali suite. It primarily focuses on web-based, open-source reconnaissance. It includes many modules, interactive help, and convenience functions that guide users to use the tool correctly. Recon-ng automates activities like cutting, pasting and harvesting, which can become really time-consuming to have to do manually. It also performs operations like database interaction, performs web requests, and manages API keys.
9. Shodan: Shodan is an online ‘find everything that is pinging on the internet’-type OSINT website. Shodan is much like a security monitor and a dedicated search engine used to find data and intelligence on the Internet of Things [IoTs]. This tool is also known as the search engine of hackers as it helps to find and explore different devices connected to a network(s). It also helps detect and find open ports and vulnerabilities on the attack surface. Shodan is of specific interest for IT professionals as it gives information and details about HTTP, SSP, SNMP, and RTSP, which are based on operating systems, countries, networks, and ports. Along with the IoT devices, Shodan can also query databases and find data publicly accessible through paths other than the main interface.
10. the harvester: The harvester is one of the simplest tools to capture and access public information outside an organization’s network perimeter. it brings back valuable information about virtual hosts, subdomain names, email addresses, and open ports of any organization. This tool is very helpful in determining the scope of the pen test and helps as a reconnaissance step before a pen test initiates. The harvester uses popular search engines like Google, Duck Duck Go, Bing, and social media networks to collect OSINT.
11. Metagoofil: As the name suggests, Metagoofil is used to extract metadata from public documents that also include PDF’s and Microsoft Office files. It finds the target document and stores it on a local disk and maps the paths used to get the documents. This helps obtain directory tree information, shared resources, and server names of the host organization. This is a perfect tool for hackers to gather information and launch brute-force attacks on the target system(s). This tool also helps cyber security professionals determine the vulnerabilities and helps to secure networks by closing the gaps before the hacker(s) exploit the vulnerabilities discovered.
12. Censys: This is a wonderful tool that acts as a search engine to get information about any device or network system connected to the internet. They can also return information on servers and domain names. In addition, you can find geoinformation and technical details about ports 80 [HTTP] and 443 [HTTPS] running on a server, HTTP mapping of the target website, SSL certificate information, TLS handshake information, and WHOIS information.
13. TinEye: TinEye is a reverse image tool and image recognition tool that mainly focuses on reverse image searches that helps moderate the content that is posted on the web and is available for access through public domains. It can detect instances of fraud and copyright occurring through images patterns recognition and track the location of these images online among the constantly growing index of billions of images online posted daily.
14. OpenVAS: OpenVAS stands for Open Vulnerability Assessment and is a security framework that includes a vulnerability scanner for it professionals to detect threats and vulnerabilities in systems. It is used for authentication and unauthenticated testing, performance tuning for scans, high-level industrial protocols, and a powerful internal programming language to carry out vulnerability tests from a continuous daily feed. It enforces security by enabling continuous monitoring of networks, systems, software, applications and hardware for threats and vulnerabilities.
15. searchcode: searchcode is a unique and dedicated search engine that searches the code repository for any intelligence inside free source code. Works like any other normal search engine, but instead of searching for indexed web servers, searchcode searches for information in the code repositories of running apps or apps in the developing stage. It is completely free, and its filters make it easy for the users to sort data by language, repository, or phrase. It is a good OSINT tool since it gathers information from accessible source codes and checks for sensitive information. searchcode is a good tool to have when the apps are in the developmental stage and can be used as a reconnaissance tool before the deployment stage.
