Module 4: HTTP and RESTful Services

Module Overview

Master HTTP methods and RESTful API concepts to build and interact with web services.

Learning Objectives

• Given a use case fulfilled by a RESTful service, classify whether its endpoint should be invoked through a GET, POST, PUT or DELETE request
• Identify the corresponding HTTP response code class for a given description of an interaction with a RESTful endpoint
• Given a RESTful API's documentation, implement submission of a GET request to retrieve a resource
• Given a RESTful API's documentation, implement submission of a PUT request to update a resource
• Given a RESTful API's documentation, implement submission of a POST request to create a resource
• Given a RESTful API's documentation, implement submission of a DELETE request to delete a resource
• Recall that to send an HTTP request, the host name must be resolved to an IP address
• Explain how ports enable a host to handle incoming requests for multiple services
• Recall that an API defines how to interact with a given service
• Define the terms HTTP, IP, DNS, port, URL
• Outline the cases in which each of GET, POST, PUT, and DELETE HTTP request methods is used
• Outline the meaning of the 2xx, 4xx, and 5xx HTTP response code classes
• Use cURL to test HTTP requests/responses
• Explain what it means for an operation to be idempotent
• Understand RESTful service design principles and best practices
• Master the use of HTTP methods for CRUD operations
• Learn how to make requests to RESTful APIs
• Explore techniques for consuming and processing API responses
• Understand status codes and their meanings in API communication

Making a Request to a RESTful API

Understanding HTTP and RESTful APIs

In the last reading, we introduced HTTP (Hypertext Transfer Protocol), a request-response protocol between a client and server, and RESTful APIs (Representational State Transfer), a design pattern used by developers to create understandable APIs. In this reading, we will discuss HTTP requests and responses and how they're used in general and in RESTful APIs.

Figure 1: Diagram representation of HTTP Request Model, showing a client sending a GET request to a service endpoint and receiving a JSON response.

You may recall this image from the previous reading. We've changed it slightly to show that communicating with a RESTful API uses the same process (HTTP) but can send back different data than communicating with a web server might. Before we start working with our own API, we're going to use the GitHub API as we work our way through understanding HTTP requests and responses.

The HTTP Message Format

To transfer data between the server and client we use HTTP messages. There are two types of HTTP messages: requests, which are sent by the client to retrieve information or trigger an action on the server, and responses, which are the server's answers.

One of the benefits of HTTP is that it provides a standard format for transferring these messages. You can see an example of the structure here in the headers for our request to the Github API:

GET /users/defunct HTTP/1.1
Host: api.github.com
Connection: keep-alive

These lines represent part of the request header. They specify the server we're accessing and what we're doing. What action we're doing is determined by the method that's used. There are four request methods that we'll be discussing in this reading: GET, POST, PUT, and DELETE (we'll explain these methods in more detail in the next section). In the first line above we see the Method (GET), the end point (/users/defunct) and the protocol (HTTP). The next line is the server we're accessing (api.github.com) and the last line is asking what type of connection we want. In this case, we are expecting a response, so the connection is kept alive in order to receive that response.

The HTTP Response

When you send out an HTTP request to the server, you'll get an HTTP response in reply. All HTTP responses include a response code that classifies what kind of response it is. These codes come in the form of a three-digit number and are broken into three primary categories:

• 2xx — Success!
• 4xx — You messed up your request.
• 5xx — The server messed up.

The most common response codes that you'll see are:

• 200 — OK
• 404 — Not Found
• 500 — Server Error

Using Curl to Test HTTP Requests/Responses

cURL or curl, which stands for client URL, is a command line tool for sending and receiving data using a wide variety of protocols, including HTTP. We'll be using it to make HTTP requests and view their HTTP responses.

The following is the curl command structure and some common flags:

curl <flags> <URL>

-G — use the GET HTTP method for the request
-d — specifies the data to pass in the body of the request, and will use the POST HTTP method for the request by default
-i — include headers in the output
-I — only show headers in the output (NOTE: this is a capital i)
-X PUT — -X changes the HTTP method based on the provided value, commonly used with -d

HTTP Request Idempotence

Another important property of certain HTTP requests is idempotence. Idempotence describes an operation that in response to the same input will always result in the system having the same end state.

The way that the GET, PUT and DELETE methods are used in RESTful services is idempotent. If we make multiple, identical GET or PUT requests, we expect the result to always be the same. On the other hand, the way that POST methods are usually implemented in RESTful services is not idempotent. If you make multiple, identical POST requests, you will not always get the same result.

HTTP Methods and RESTful API Requests

HTTP methods define the type of operation being performed on a resource in a RESTful API. Each method has specific semantics and use cases:

GET

Used to retrieve resources without modifying them. GET requests are:

• Read-only - They should never modify data
• Idempotent - Making the same request multiple times produces the same result
• Cacheable - Responses can be cached for performance

Example of a GET request using Java's HttpClient:

HttpClient client = HttpClient.newHttpClient();

HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create("https://api.example.com/songs/imagine-dragons/radioactive"))
    .header("Accept", "application/json")
    .GET()  // This is the default, so it could be omitted
    .build();

HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());

System.out.println("Status code: " + response.statusCode());
System.out.println("Response body: " + response.body());
                

POST

Used to create new resources. POST requests are:

• Not idempotent - Multiple identical requests may create multiple resources
• Not cacheable - Generally, responses should not be cached
• Includes a request body - Contains the data for the new resource

Example of a POST request:

PUT

Used to update existing resources or create them if they don't exist. PUT requests are:

• Idempotent - Multiple identical requests have the same effect as a single request
• Complete replacement - The entire resource is replaced with the request payload
• Requires the complete resource - All fields should be provided

Example of a PUT request:

DELETE

Used to remove resources. DELETE requests are:

• Idempotent - Multiple identical requests have the same effect (resource remains deleted)
• Typically don't include a request body - The resource to delete is identified by the URL

Example of a DELETE request:

HTTP Status Codes

HTTP status codes indicate the result of the request:

• 2xx (Success) - The request was successfully received, understood, and accepted
  • 200 OK - Standard success response
  • 201 Created - Resource successfully created
  • 204 No Content - Success with no response body
• 4xx (Client Error) - The request contains errors or cannot be fulfilled
  • 400 Bad Request - Invalid syntax or parameters
  • 401 Unauthorized - Authentication required
  • 403 Forbidden - Server understands but refuses the request
  • 404 Not Found - Resource doesn't exist
  • 409 Conflict - Request conflicts with server state
• 5xx (Server Error) - The server failed to fulfill a valid request
  • 500 Internal Server Error - Generic server error
  • 502 Bad Gateway - Invalid response from upstream server
  • 503 Service Unavailable - Server temporarily unavailable

Consuming a RESTful API

Consuming a RESTful API

Take a small break and just observe on this one. The original link is down, however the concepts we cover here can be used in many situations.

In the previous two readings, you learned about how RESTful APIs use HTTP requests and responses. In this reading you will look at a mock social media RESTful API we have created and learn about consuming a RESTful API.

To consume an API means to make use of it in the program you are writing. The ease of consuming an API largely relies on the APIs documentation. API documentation is important because it contains information for the developer about how to effectively use and integrate the API.

Social Media API Documentation

We want to help you create your own social media web app for your friends. We've gone ahead and built a mock RESTful API to represent this social media app and give you some experience with API documentation.

Our API supports the ability to create a new user, create statuses, see other users and their statuses, and edit and delete statuses. To support this functionality there are two GET methods, two POST methods, one PUT method, and one DELETE method.

You can find the API documentation to reference in Warmup 04 - Social Media API Documentation.

GET status

To help us become familiar with reading API documentation we're going to walk through the GET status method.

This method is an HTTP GET method and its purpose is to return the data from a status. You would use this method any time you want to display a certain status, which could be on a status feed or on a user's profile.

In the documentation, the relevant endpoint is described as the Resource URL.

https://nzbzh3a24d.execute-api.us-east-1.amazonaws.com/socialmediaexample/status (link currently down)

This describes the path that is used to access the requested resource and must be present in the curl command. Note that this URL is not accessible, but it will be used in the examples below.

There are two parameters listed: userId and statusId. Both are tagged as 'required', which means the request will only be accepted by the server with both parameters. Otherwise, you will receive an error message. The examples in the documentation are userId=1 and statusId=1 which must be added to the endpoint described above. A question mark is appended to the endpoint and the parameters are combined using an ampersand, &, as shown: ?userId=1&statusId=1. Providing data this way is sometimes referred to as URL parameters since the data is a part of the URL.

The fully formed request:

curl -G "https://nzbzh3a24d.execute-api.us-east-1.amazonaws.com/socialmediaexample/status?&statusId=1&userId=1"

The response is returned as one JSON object as described in the resource information, given below.

{ "status": { "userId": 1, "statusId": 1, "message": "About to test a REST API, wish me luck!" } }

Notice that the user id and status id are returned in the JSON format that matches the input parameters. The other variable belonging to the JSON object is the user's status message, "About to test a REST API, wish me luck!".

Now let's put this command in the terminal and watch the result,

ATA-Mac:~ ata-user$ curl -G "https://nzbzh3a24d.execute-api.us-east-1.amazonaws.com/socialmediaexample/status?&statusId=1&userId=1" { "status": { "userId": 1, "statusId": 1, "message": "About to test a REST API, wish me luck!" } }

The following sequence diagram shows the steps that take place when we make the above curl command.

Figure 1: Sequence diagram of a curl request to a GET endpoint. Human-readable diagram source

POST status

We're also going to walk through the POST status method to discuss the method's authentication information.

This method is an HTTP POST method and the purpose of it is to create a new status and populate it with the given data. Social media platforms often allow anyone to view statuses, therefore you don't need authentication for the GET requests. However, in order to create a status, you need to be logged in to your account. The POST status method uses HTTP authentication.

Authentication is the process of providing a login and password to a server that will validate that you are allowed to make the request. Instead of a plain text password, it is best practice to authenticate using hashed values that represent your account information, called an authentication token. This process is similar to hashing an object's data to store in a HashSet. There are several authentication methods that can generate this information and are used in different ways. Each authentication method has a standard that has been developed and is publicly available so that developers don't have to design their own authentication methods, they may choose one that already exists. This API uses a method called OAuth, and will be shown below.

In the documentation the relevant endpoint is described as the Resource URL.

https://nzbzh3a24d.execute-api.us-east-1.amazonaws.com/socialmediaexample/status (Link currently down)

This describes the path that is used to access the requested resource and must be present in a curl command. Notice how the endpoint is the same as the GET method for status. The server will know which action you are performing by the HTTP method used, in this case POST.

There are two parameters listed, userId and status. They are both tagged as 'required', which means the request is only valid with the parameters, otherwise, you will receive an error message. The examples in the documentation are userId=1 and status=Hello which must be added to the endpoint described above. A question mark is appended to the endpoint, and the parameters are combined using an ampersand, &, as shown: ?userId=1&status=Hello.

Note that this is one way to provide data in an HTTP request, we will explore other ways of passing data such as in the body of the request later in this unit.

This method also requires authentication through OAuth. For our purposes, the example values listed below will work. This lesson will not cover how to generate an OAuth token or the specifics behind what each value means, but this lesson will provide sample data that is used correctly in standard OAuth authentication. You are not expected to know this at this time. Each key-value pair must be appended to the curl command with the --header notation:

--header oauth_consumer_key:"0685abc22" --header oauth_token:"ad180lru7" --header oauth_signature_method:"HMAC-SHA1" --header oauth_signature:"wOJIO9A2W5mF" --header oauth_timestamp:"137131200

The fully formed request uses the -X flag combined with "POST". This allows us to make a POST request without needing to provide data with the -d flag, since we are providing it through the parameters:

curl -X POST "https://nzbzh3a24d.execute-api.us-east-1.amazonaws.com/socialmediaexample/status?userId=1&status=Hello" --header oauth_consumer_key:"0685abc22" --header oauth_token:"ad180lru7" --header oauth_signature_method:"HMAC-SHA1" --header oauth_signature:"wOJIO9A2W5mF" --header oauth_timestamp:"137131200" --header oauth_nonce:"4572616e48" --header oauth_version:"1.0"

The example response is listed and returned as one JSON object as described in the resource information.

{ "status": { "userId": 1, "statusId": 2, "message": "Hello" } }

Notice the user id and status is returned in the JSON format that matches the input parameters. The other field in the JSON object is the user's status id, which was created upon completion of the method.

Now let's test out this command in the terminal by posting a new status with the message "Hello".

ATA-Mac:~ ata-user$ curl -X POST "https://nzbzh3a24d.execute-api.us-east-1.amazonaws.com/socialmediaexample/status?userId=1&status=Hello" --header oauth_consumer_key:"0685abc22" --header oauth_token:"ad180lru7" --header oauth_signature_method:"HMAC-SHA1" --header oauth_signature:"wOJIO9A2W5mF" --header oauth_timestamp:"137131200" --header oauth_nonce:"4572616e48" --header oauth_version:"1.0" { "status": { "id" : 1, "statusId" : 2, "message" : Hello } }

Summary

You've seen one example of a REST API and learned how to use it. Although other API's will be set up differently, there should always be documentation and guidelines for using them. They all follow the basic principles of making HTTP calls to a server and receiving responses. What you are trying to accomplish will determine what resource and method you need to call. The API reference is a great place to start when dealing with new APIs as it will have valuable information on how to use them. The key things to look for are what endpoint needs to be called, what HTTP method, the parameters, the response format, and whether authentication is required.

Processing API Responses

When consuming RESTful APIs, you need to properly handle and process the responses:

Response Formats

RESTful APIs typically return data in JSON or XML format. JSON is the most common format due to its simplicity and compatibility with JavaScript:

Parsing JSON Responses

In Java, you can parse JSON responses using libraries like Jackson or Gson:

Error Handling

Proper error handling is crucial when working with APIs:

Testing with cURL

cURL is a command-line tool for making HTTP requests, useful for testing APIs:

HTTP and RESTful Services Overview

RESTful API Design Best Practices

When designing or working with RESTful APIs, keep these best practices in mind:

1. Resource Naming

• Use nouns, not verbs - Resources should be named as nouns, not actions
• Use plurals - Collections should be plural (e.g., /songs, not /song)
• Use lowercase - All URLs should be lowercase
• Use hyphens for readability - Use hyphens (-) instead of underscores (_)

2. Resource Hierarchy

3. Use HTTP Methods Correctly

• GET for reading (never for modifying)
• POST for creating
• PUT for complete updates
• PATCH for partial updates
• DELETE for removing

4. Always Use HTTPS

Secure your API with HTTPS to protect data in transit. This is not optional in production environments.

5. Versioning

Include API versioning to make changes without breaking existing clients:

6. Use Proper Status Codes

Return appropriate HTTP status codes for different scenarios:

• 200 OK for successful GET, PUT, or PATCH
• 201 Created for successful POST
• 204 No Content for successful DELETE
• 400 Bad Request for validation errors
• 401 Unauthorized for missing authentication
• 403 Forbidden for insufficient permissions
• 404 Not Found for non-existent resources
• 500 Internal Server Error for server-side issues

7. Pagination, Filtering, and Sorting

For collections, support pagination, filtering, and sorting via query parameters:

8. Error Handling

Return descriptive error responses with appropriate status codes:

9. Idempotency

Understand and respect idempotency:

• Idempotent operations produce the same result regardless of how many times they're called
• GET, PUT, DELETE are idempotent
• POST is not idempotent

This is especially important for error handling and retries. If a request fails due to a network issue, idempotent operations can be safely retried without side effects.