During our engagements, we sometimes find customers have difficulty in determining what hosts they own and if they are live on the Internet. This can easily happen when you have a high turnover of networking staff, where the knowledge is not passed on, or you have a large infrastructure presence that can make it difficult to constantly manage/monitor. In a worst-case scenario, this can lead to compromise of data and possible exploitation of your internal network where ‘forgotten’ hosts are left unpatched and unmanaged.
Identifying and Securing Forgotten Hosts in Your Network
This obviously would be a concern when the customer is required to define the scope of, say, an external penetration test. These ‘forgotten’ hosts are not included and therefore do not get tested.
If this rings a bell, or it may be something you take for granted, then the following helpful tips may prove useful when confirming your external presence on the internet.
Step 1 - Determine your hosts and networks
If you aren’t sure what hosts your corporation owns, here’s a few helpful tips that you can use (or a potential hacker may use):
- Ask your service provider – It may sound too obvious, but if you are paying for hosts on the internet or cloud services, your Internet Service Provider (ISP) will know what IP’s or network range(s) you are paying for. If you are the company concerned, this should be your first stop - follow the money!
- Query Autonomous System Number (ASN) – The ASN number is a unique number assigned to a large corporation or service provider that own one or more Autonomous Systems in its WAN infrastructure. It is typically used within the Border Gateway Protocol (BGP) routing protocol which allows entities connected to the Internet to communicate using their unique ASN number in the BGP configuration. We can find the ASN by querying research sites’ data, for example:
http://download.maxmind.com/download/geoip/database/asnum/GeoIPASNum2.zip and then searching for the company name with their data. Once we have the ASN number we can then query DNS using tools like ‘whois’ supplying the AS number (e.g. “whois -h whois.radb.net -- '-i origin AS12345' | grep -Eo "([0-9.]+){4}/[0-9]+"
- Query Internet Registrar - A domain name registrar is an organization that manages the reservation of Internet domain names. We can use tools such as ‘whois’ to query multiple whois databases on the internet. We may need to query the correct registrar depending on the information we are after. The following are some examples:
Find IP Allocations - provide IP allocations dependent on region:
- www.arin.net – covers North and South America and sub-Saharan Africa. We can use a wildcard to find all IP’s allocated to a company as follows e.g. whois "company*" -h whois.arin.net
- www.apnic.net – covers Asia Pacific
- www.ripe.net – covers Europe, Middle East and some of Africa
- www.afrinic.net – covers Africa
Find Domain Names – search for domains containing the query string (in this case the company name) ending in .com. g. whois "company*" -h whois.crsnic.net
- Google Hacks/Dorks – You can use Google’s search engine to only list results directly related to the target domain by using the ‘site’ directive. Try googling this example: site:company.com
This will return a list of hits within the company.com domain. To narrow down the search we can use the ‘-site’ directive and remove the already discovered or accounted for URLs and continue refining the result set for example: site:company.com -site:www.company.com -site:dev.company.com
And so on, until there are no more websites discovered. We can then use the list of websites discovered using nslookup to find their IP addresses and then run a whois query on the IP to get the network block and other information.
- Query Censys – After free registration, Censys is a search engine that collects data on millions of hosts on the Internet and allows you to search the data for information. For example, you can enter the company name and see what IP’s are returned. It also scans for common services such as SMTP, Telnet, and HTTPS and collates the data. This is also a good method of passively finding out the services running on a host. There is also an API to write your own tools to query the data.
- Query Robtex - Robtex uses various sources to gather public information about IP addresses, domain names, host names, autonomous systems, routes etc. It then indexes the data in a big database and provides free access to the data.
- DNS Brute Force – If you know the domain name, you can perform an automated brute force discovery to look for common host names from a suitable dictionary. So if the domain is company.com then our dictionary would have a list of words such as www, dev, mail,smtp,backup etc and our tool would then perform an nslookup on www.company.com, mail.company.com, smtp.company.com and backup.company.com. If the host name resolves to an IP, then we can perform a whois search on that IP to look for the network block.
- Spider Known Sites – For websites you have discovered belonging to your company, it’s also good to know what other sites it has links to. This can expose other hosts may communicate with that site which you are unaware of. Tools such as Burp suite (https://portswigger.net/burp) and Xenu Link Sleuth. (http://home.snafu.de/tilman/xenulink.html) allows you to spider the site and review the results.
- Zone Transfers – Not so common any more, however you may still be able to download all host names configured within your authoritative DNS name server.
There are services and tools out there that will charge you for this service, however my advice would be to see how far you get using the above free resources before handing over any money. Also worth noting, some of the information will not be in the public domain, in which case go back to the first pointer and “follow the money” I.e. what do you pay for?
The following simple script performs some of the above queries. You could make this as complicated as you want by querying other registrars, but you should get the idea:
#!/bin/bash
# Script to attempt to find IP's and networks belonging to a company
#
ARGS=2
CORP=$1
LIST=$2
RED='\033[0;31m'
NC='\033[0m' # No Color
IFS=$'\n'
######################################
if [ $# -ne "$ARGS" ]; then
printf "Usage: `basename $0` <Company Name> <DNS names file>\n"
echo "Attempts to find IP's and domains belonging to company."
echo "Also supply a list of possible host names for DNS checks"
exit 0
fi
echo ""
printf "${RED}Attempting to find the ASN Number for the company $CORP${NC}\n"
asn_array=( `curl -s http://download.maxmind.com/download/geoip/database/asnum/GeoIPASNum2.zip | gunzip | cut -d"," -f3 | sed 's/"//g' | sort -u | grep -i $CORP` )
asn_array_length=${asn_array[@]}
echo "Number of ASNs = $asn_array_length"
if [ $asn_array_length = 0 ]; then
echo ""
echo "Company $CORP does not have an ASN number"
else
echo ""
for asn in "${asn_array[@]}"
do
echo "${asn}"
done
fi
echo ""
if [ $asn_array_length != 0 ]; then
echo ""
printf "${RED}Finding Networks belonging to $CORP's ASNs:${NC}"
echo ""
for i in "${asn_array[@]}"
do
asn_no=( `echo ${i}|cut -d" " -f1` )
network=( `whois -h whois.radb.net -- "-i origin ${asn_no}" | grep -Eo "([0-9.]+){4}/[0-9]+"` )
echo "$network belongs to ${asn_no} "
done
fi
echo ""
printf "${RED}Attempting to find other domains with $CORP in name:${NC}"
echo ""
echo "`whois "$CORP*" -H -h whois.crsnic.net `"
echo ""
printf "${RED}Attempting to find IPs allocated to $CORP from arin.net:${NC}"
echo ""
echo "`whois "$CORP*" -H -h whois.arin.net `"
echo ""
printf "${RED}Attempting to find hosts belonging to $CORP.com domain:${NC}"
echo ""
NS=$(dig +short SOA $CORP.com|cut -d' ' -f1)
for server in $(cat $LIST); do
host $server.$CORP.com $NS|grep 'has address \| is an alias'
done
Step 2 – Next Steps
After having gathered as much information from research and reconnaissance the next steps should involve ping sweeping, port scanning, vulnerability scanning and penetration testing the hosts discovered to ensure your entire external network infrastructure is secure. This whole process should be carried out on a continual regular basis to ensure that you are confident that all your hosts you are responsible for are tested and remain secure.
If you'd like to discuss your security posture and see how we can help you in your journey to a more secure network, please feel free to contact us! We provide a number of cybersecurity related services, PCI Compliance, P2PE, PIN, web security solutions, pen testing and threat detection to name just a few!