This is one part of a series of posts on how to prepare your API for a pentest. The first post was focused on Insomnia. The second was focused on Postman. Check back in the near future for additional content.
Similar to web applications, web APIs (Application Programming Interfaces) should undergo security testing to determine whether or not any vulnerabilities exist. However, an API may not be as straightforward to test as a web application. It may not be possible to provide a URL to a pentester and say test everything underneath this. As such, pentesters will ask for test data and the ability to access the API for security testing. In this post, we will focus on using the curl program to provide data.
Curl (short for Client URL), is a ‘command-line tool for transferring data specified with URL syntax.’ It can be used to connect to web applications and APIs to pull down data. It is often used in a scriptable environment and developers can use it to easily call APIs on a regular or automated basis. One of the great strengths of curl is it has an extensive history and can be used on nearly every operating system on the market. If it is not already included in your operating system, you can find a copy at https://curl.haxx.se/download.html.
Curl is a widely used command line tool that can call web applications and APIs. With some minimal modifications, curl can be used to send API calls into various Intercepting Proxy tools (such as Burp or OWASP ZAP). Using pre-built test data will greatly speed up the pentesting timeframe, often lowers the pentest project cost, and provides better pentest report quality. From there, pentesters use the Intercepting Proxy to perform various active and manual testing by interacting with the API directly. Oftentimes curl does not need to be used again after performing the initial API call.
If you already use curl within your environment and want to provide the data for pentesting, please scroll down to the ‘Exporting Curl Data’ section.
Curl has a lot of customization if you want to dig into the documentation, but our main concern about is how to create the smallest functional request possible. We are going to go through some of the basic components and build a simple curl request for three REST API actions – one to fetch a ‘to do’ list, one to post a comment, and finally one to delete a comment. If you want to try curl out in your own environment, use https://jsonplaceholder.typicode.com as a practice API server.
To start, open a terminal window. Try calling the https://jsonplaceholder.typicode.com URL with no modifications using the following command:
curl https://jsonplaceholder.typicode.comWe can see that curl calls the main website and displays it as text, but that did not work to interact with any APIs. If the curl command was not found, use https://curl.haxx.se/download.html to download a version of curl that would work for your operating system.
Next we want to call our ‘to do’ API to get our results. Given that it’s just a REST API, all we need to do is append ‘/todos’ within the URL.
curl https://jsonplaceholder.typicode.com/todosAs an owner of the application, we may know that multiple methods or additions can be added to our API to get specific data. For example, we can append /3 to request data attached to ID #3.
curl https://jsonplaceholder.typicode.com/todos/3This works well for performing HTTP GET operations, as curl sends a GET by default. What do we do in order to build a POST request? For this next example, we do add a few extra curl flags to our command that allows us to post a comment to the site. The -X flag allows us to specify the use of the POST method, and the -d flag allows us to specify the content in the body. Be sure to put quotes around the body content!
curl -X POST -d “test”
https://jsonplaceholder.typicode.com/postsWe can see with the above screenshot that we were able to successfully create a new comment via the POST command.
Finally, what can you do regarding authentication? In most cases, this just requires you to add a custom header into your request. For example, we know that in order to delete a comment via the API, we are required to use the DELETE method and provide authentication. If you know your header needs to be listed as ‘Authentication: Bearer [token]’, we can add the -H command with the header contents in single quotes.
curl -X DELETE -H ‘Authentication: Bearer TG9va2luZ1RvSm9pblVzP0dvVG9odHRwczovL3d3dy53aGl0ZW9ha3NlY3VyaXR5LmNvbS9jYXJlZXJz’ https://jsonplaceholder.typicode.com/posts/103We can see the site successfully responds to the API call by providing a blank response.
As a note, some terminal windows will normally accept double quotes vs single quotes. You may need to switch those around to find what works best in your terminal environment.
Copy two valid example curl commands for each API into a text file. Make sure that search functions or specific parameters are treated as a ‘unique’ API.
Using the previous section as an example, if you have a total of four (4) unique APIs, copy a total of up to eight (8) maximum examples into a text file. Some APIs only have a single function with no extra test data, so it is not possible to provide additional examples. It should be line separated (and maybe with some helpful descriptive text) between each example like the following screenshot:
Once this text file is completed, coordinate with your pentester to send the file. Alongside any updated credentials or session information (such as authorization tokens), this might be all that is needed to ensure a successful API pentest!