Reviewing Automated HTTP Screenshot Tools
In this Screenshot Tool blog series, our Cyber Advisors technical team will be reviewing the efficacy of the top HTTP screenshot tools that are currently available for penetration testers or bug bounty hunters. Each blog post will cover the available features within the tool, how easy it is to use, and any problems that may arise while performing simple scans. At the end of this series, we’ll put all the tools to the test and compare their effectiveness against a set of metrics to find the most useful tool.
Screenshot Tools
We use HTTP screenshotting tools on external, internal, and red team engagements to quickly analyze the footprint of web servers and services exposed across the target environment.
The following items represent the criteria for how I define ‘usefulness’ for a given HTTP Screenshot Tool. These criteria are skewed towards attack surfacing reconnaissance and penetration testing techniques.
Note: this series was written in Feb. 2022.
Sources & Customization
- Are common network scanner output formats supported as input sources by the tool? (Nmap/Masscan/Nessus)
- Can we append a second scan to the results of the first scan within the Database?
- Can you set a custom User-Agent?
- Can you set custom request headers and/or cookies?
Performance
- How accurate is the tool? How many errors does it generate? Does it successfully capture screenshots (no white or blank screenshots)?
- How quickly does the tool perform the task? Can we speed the scan up without degrading its accuracy of the scan?
- Can we easily proxy the network traffic to http and socks proxies?
- Can we provide a navigation timeout against slow assets?
Reporting
- Does the tool group or categorize similar hosts based on their screenshots?
- Does the tool provide useful output? Can it export to PDF/HTML/JSON/CSV?
- Is the UI/UX functional and enjoyable to use?
General
- Is the tool actively maintained with updates?
- Any Cross-Platform compatibility?
- How easy is it to build and install the project?
EyeWitness
Now that we have defined our success criteria, we will start our tool comparison series with one of the most well-known HTTP screenshot tools: EyeWitness. The project’s description is as follows: “EyeWitness is designed to take screenshots of websites, provide some server header info, and identify default credentials if known.”
Notable Features
EyeWitness provides several key features which have helped establish its place within the repertoire of penetration testers. The most interesting feature is a pattern of signatures that are applied to each screenshot for the detection of common network appliances and devices and their default credentials. This feature is probably the main reason a lot of people have relied on EyeWitness in the past since it helps cut down on the time it takes to google the appliance name and find default credentials within support files.
A snapshot of the CLI arguments can be seen below which provides a window into the tool’s feature set.
EyeWitness Sources & Customization
EyeWitness appears to check a lot of the boxes in our criteria lists such as providing custom User-Agents, proxy support, multiple threads, and network timeout capabilities. EyeWitness also provides the capability to parse Nmap, Nessus, and flat files.
EyeWitness Reporting
The EyeWitness report provides several useful features, the most notable being that EyeWitness attempts to categorize similar hosts as well as categorize based on errors/response codes/login pages, and potential credentials for hosts. This is a handy reporting feature, as you’re able to easily identify similar applications. The report also supports pagination, which means that the screenshots are neatly segmented across multiple HTML files rather than being dumped into one HTML file which can cause memory exhaustion.
On the other hand, the report does not have the ability to search within the report UI and the HTML template is rather large, meaning it takes quite a bit of time to scroll through thousands of hosts.
EyeWitness General Info
EyeWitness’s setup is fairly light, most notably needing to download the Mozilla geckodriver and download several python dependencies (such as Selenium). It would be nice to see installation instructions out of the box for Python’s virtualenv and/or pipx, to isolate the build.
EyeWitness provides a dedicated Windows binary build for cross-platform execution. The EyeWitness project is actively maintained at the time of writing, with recent code changes throughout the year 2021. EyeWitness supports threading and allocates its resources across all available CPUs.
Out-Of-The-Box Tests
Now that we’ve covered some of the high-level features of EyeWitness, it’s time to actually try the utility out. Installation was easy enough following the setup instructions, and building EyeWitness for Debian. However, I had run into issues with the EyeWitness tool.
Under our criteria for a useful screenshot tool, an important requirement is to be able to proxy traffic to a SOCKS server. While EyeWitness appears to have that capability built-in, I was not able to get EyeWitness to successfully generate a report using a socks proxy. Perhaps this was just an issue with my environment, however, this same SOCKS tunnel worked perfectly with every other tool in this blog series.
In troubleshooting EyeWitness’s SOCKS proxy, I was able to generate a sample report successfully without using the SOCKS proxy:
However, without using the socks proxy on my host machine, EyeWitness would hang and I had to physically kill the Firefox process tree and the EyeWitness python processes to terminate the running job.
Manually terminating the root EyeWitness python processes does not clean up the remaining open browsers, resulting in zombie firefox instances. Running EyeWitness through a SOCKS proxy directly on an AWS EC2 debian instance resulted in similar (albeit, worse) performance with processes hanging.
When I ran the performance test through a SOCKS proxy, EyeWitness would successfully capture the screenshots, however, it would not generate a successful report. I tried installing EyeWitness on an Ubuntu virtual machine with a different SOCKS proxy and I observed the same behavior.
The screenshots were successfully captured on disk, but maybe there is a bug within the EyeWitness code base that results in a false positive error signal when proxying traffic. EyeWitness does not have any debug arguments built-in, so additional troubleshooting would be needed to figure out the root cause of this issue.
Best Performance EyeWitness Setup
The following is the best possible screenshot capture configuration based on EyeWitness’s built-in arguments that allowed for the highest success percentage of captures during my analysis:
- Custom User-Agent (–user-agent)
- Navigation Timeout (–timeout 20)
- Navigation Delay (–delay 8)
- Ten Threads (–threads 10)
End Of Part 1
Overall, EyeWitness provides a healthy feature set and it is clear why this tool has stuck around for many years as part of a pentester’s toolkit. The benefits of having default credentials and simple categorization are really nice in some people’s eyes, though I do wish there was an easier way to manipulate the data within the HTML report such as sorting, searching, and note-taking.
I, personally, could not coax EyeWitness into running a successful scan through a SOCKS proxy. I’ve tried as many different environments as I have the patience for (Ubuntu vs Debian, local OS vs cloud OS). Historically, I’ve had this issue as well where EyeWitness would either fail or take an extremely long amount of time to perform its job. At least the screenshots were captured on disk, even though the report generation failed, which allows for some analysis to be performed in a more traditional fashion.
At the end of this Screenshot Tool series, we will dive deep into a performance comparison of EyeWitness compared with its competitors.
If you use EyeWitness as part of your toolkit and would like to see new features and bug fixes in the future, please support and contribute to this Open-Source project.
Check out Part 2 – WitnessMe, as our series continues.
Author’s Note: This blog series is my personal take on the state of screenshot tools. My network environment and physical setup may differ from yours when using these tools and you may notice your tool is more or less successful. I’ve done the best I can to be platform agnostic to provide the best environment for success. If you run into any errors, remember to check the individual project’s issues page for support.
MORE FROM OUR TECHNICAL BLOG
Cyber Advisors specializes in providing fully customizable cyber security solutions & services. Our knowledgeable, highly skilled, talented security experts are here to help design, deliver, implement, manage, monitor, put your defenses to the test, & strengthen your systems - so you don’t have to.
