Unlock exclusive free benefits that come with your purchase, thoughtfully crafted to supercharge your learning journey and help you learn without limits.

Here’s a quick overview of what you get with this book:

Next-gen reader

Figure 1.1: Illustration of the next-gen Packt Reader’s features

Our web-based reader, designed to help you learn effectively, comes with the following features: Multi-device progress sync: Learn from any device with seamless progress sync. Highlighting and notetaking: Turn your reading into lasting knowledge. Bookmarking: Revisit your most important learnings anytime. Dark mode: Focus with minimal eye strain by switching to dark or sepia mode.

Interactive AI assistant (beta)

Figure 1.2: Illustration of Packt’s AI assistant

Our interactive AI assistant has been trained on the content of this book, to maximize your learning experience. It comes with the following features: Summarize it: Summarize key sections or an entire chapter. AI code explainers: In the next-gen Packt Reader, click the Explain button above each code block for AI-powered code explanations. Note: The AI assistant is part of next-gen Packt Reader and is still in beta.

DRM-free PDF or ePub version

Figure 1.3: Free PDF and ePub

Learn without limits with the following perks included with your purchase: Learn from anywhere with a DRM-free PDF copy of this book. Use your favorite e-reader to learn using a DRM-free ePub version of this book.

Unlock this book’s exclusive benefits now Scan this QR code or go to packtpub.com/unlock, then search for this book by name. Ensure it’s the correct edition.
Note: Keep your purchase invoice ready before you start.

All the code examples for this chapter are included in this book’s GitHub repository: https://github.com/PacktPublishing/Modern-REST-API-Development-in-Go. You only need the Go compiler installed on your system to compile and execute the code.

One of the first questions we should answer here is, what’s an API? As mentioned in the introduction, it’s an application programming interface, but what does that even mean?

In general terms, it’s an interface for programmers to interact with an application. However, there are different definitions of an API, depending on the context. One of them could be the set of methods/functions that are exposed by a library. That’s known as a library API. Of course, as a software engineer, you’re dealing with these kinds of APIs all day long – your Go packages, your exported functions, and the standard library are all APIs, but they focus on in-process communication, organizing your code, and separating concerns. But, in modern systems, when we talk about APIs, we’re usually talking about remote APIs.

So... what’s a remote API? A remote API is a set of functionalities that are exposed to other processes, generally through a communication channel (such as a socket), allowing independent programs to interact with each other.

But that’s still abstract, so let’s be more specific about what we’ll discuss in this book. We’ll talk specifically about REST APIs, one of the most common ways of inter-process communication used today. In REST, the communication channel that’s used is an HTTP connection that uses the HTTP protocol, so the API behaves like a web server and uses the same technology to communicate with other processes.

Before we explore REST and why we chose it over other options (maybe the best option for you is a different one), let’s explore the most popular options that are available.

GraphQL

GraphQL is one of the modern approaches to providing APIs. It was born from the necessity to provide more flexible APIs to consumers without enforcing a rigid structure (REST APIs are normally less flexible for consumers). It has a lot of exciting advantages, such as the ability to request only specific fields or embed substructures in the same response on demand by asking the client for it. That’s all doable in REST, but it isn’t standardized like it is in GraphQL. GraphQL is a great option when many clients access your data in very different ways and patterns, and you must provide that flexibility. Also, GraphQL provides lots of value when you don’t know how the consumer will query your data.

gRPC

gRPC Remote Procedure Calls (gRPC) is also one of the newest approaches for creating APIs. It’s widely used in Go and has a lot of advantages. One of the main advantages is that it’s agnostic from the language, and most of the code for serialization/deserialization and remote calls is automatically generated by the tools available. The degree of communication performance in gRPC is outstanding for multiple reasons, but mainly due to its binary protocol and usage of HTTP 2 as the transport layer. gRPC is ideal for the internal communication of services/microservices but isn’t widely used in things such as web applications or public APIs. This is because HTTP 2 is a requirement and developers are normally more familiar with other technologies, such as REST.

SOAP

Simple Object Access Protocol (SOAP) is a very old technology that uses similar principles to gRPC. It uses HTTP as a transport layer, and the protocol is 100% XML. The main problem with SOAP is that compatibility between libraries is suboptimal. SOAP works well when you use the same library and language in all the services that you’re communicating with, but not so well when you have a heterogeneous environment. Also, web clients rarely use SOAP as a communication protocol.

Custom API

You can build a custom API mechanism, selecting how you serialize and deserialize the data, transport it, and define the actions you allow. You could define a whole protocol and use TCP to communicate directly, but that’s normally not needed, and there are a lot of problems that have already been solved in the previously explained technologies and, of course, in REST.

I’m adding this option for completeness, but unless you know why you’re deciding to go with this option, you should probably be using one of the existing well-known solutions.

In summary, there are multiple mechanisms for generating APIs and communicating with applications, but in this book, we’ll be using REST. So, what’s REST, and why should you use it?

Representational State Transfer (REST) is one of the most common ways of building remote APIs in modern software, especially on public APIs that need to be consumed by third-party services/clients. The main reasons why REST is so widely used are the simplicity of the concepts it embraces and its flexibility to adapt whenever those concepts don’t 100% apply to your use case.

I’m going to take a pragmatic rather than a formal approach to describing REST. If you want a more formal definition of REST, you can go to Wikipedia’s REST page and follow some of the references there. So, let’s jump into the concepts.

REST concepts

In REST, there are resources and entities. An entity is an instance of a resource. For example, “users” is a resource that represents the users in the system, but “John Doe” is an instance of that resource, so “John Doe” is an entity. Every resource and entity has a unique way of being identified in the context of REST applications – that is, via a Uniform Resource Identifier (URI). This is the URL that you use to access the resource or entity. For example, resource users could have a URI of https://example.com/api/v1/users, whereas the “John Doe” entity could have a URI of https://example.com/api/v1/users/42.

Now that we know what resources and entities are and how we can identify them by their URIs, we need a way to represent their data. How is the “John Doe” entity data represented?

REST doesn’t impose any form of data representation, but the communicating applications need to at least be able to understand the same representation. The reality is that the de facto standard that’s used for representing entities in REST is JavaScript Object Notation (JSON). In some scenarios, you may wish to select a different representation for whatever reason (for example, for performance or compatibility purposes). Other common options are MessagePack, Binary JSON (BSON), and XML.

Now that you understand the basic concepts of entities, resources, URIs, and representation, you have a solid foundation for building applications around REST. However, it’s also important to understand how we operate in those concepts and how we communicate them.

So, let’s talk about how REST leverages HTTP to communicate the operations that we want to apply to resources and entities.

REST and HTTP

Most REST APIs are based on the HTTP protocol and leverage multiple features.

In HTTP, the URL is the URI of a resource or an entity. You can put it in your browser or call it through your code, but at the end of the day, it’s the same concept: it’s a URL that represents a unique resource or entity in REST.

Once you’ve identified the resource or the entity and its URL, you must decide what you’re going to do. What action are you going to take? Are you going to delete the entity? Are you going to create a new one? Are you going to get an existing one?

The way we express the action that we want to execute in REST is done through HTTP verbs. The HTTP protocol defines a set of verbs or actions that you can execute on a URL; these are known as methods in HTTP.

There are multiple methods in HTTP, including GET, POST, and DELETE, and each has an action associated with it in the context of REST, depending on whether it’s a resource or an entity. Let’s take a look:

• REST resource (for example, /users):
  • GET: List the entities of the resource
  • POST: Create a new entity for this resource
  • DELETE: This is normally not accepted, but it can mean removing all the entities of the resource
  • PUT: Normally not accepted
  • PATCH: This is normally not accepted, but it can mean partially updating all the entities of the resource
• REST entity (for example, /users/123):
  • GET: Get the entity’s information
  • POST: Normally not accepted
  • DELETE: Delete the entity
  • PUT: Update the entire entity
  • PATCH: Partially update the entity

Once we have a way to identify resources and entities and execute actions on them, we need a way to pass information in some instances. For example, if I want to create a new entity, I need information about that entity. If I’m creating a user, I need their username, email, name, surname, and any other information that I want to store.

For that, we can use the HTTP body of the request. This body needs to be serialized in a certain format – as I mentioned previously, the de facto standard for serialization is JSON – and we also need to communicate to the server that we’re using that serialization mechanism. To do so, we can use the Content-Type: application/json header. So, the combination of the Content-Type header and the request body is enough for the client to convey the necessary information to the server.

The server now knows what action the client wants to execute – and with what data – due to the method in the request being serialized and included in the body of the request. However, we need a way to respond to that. This can also be handled using the HTTP protocol.

HTTP provides two things that are handy in this case: the response body (similar to the request body) and the status code. In REST, the response body will contain the information you’re requesting, as well as the entity or the list of entities. The status code gives you information about what happened with the operation itself.

It’s very important to pay attention to the status code because it gives you information about what happened with your request, and maybe you need to take action and handle something differently. The following are some of the most common HTTP status codes:

• 200 OK: Everything went well
• 201 Created: A new entity has been created
• 202 Accepted: The request has been accepted but hasn’t necessarily been processed
• 204 No Content: Everything went well, but no data is available
• 301 Moved Permanently: Permanent redirect
• 302 Moved: Temporary redirect
• 304 Not Modified: The data hasn’t changed (used for caches)
• 400 Bad Request: The client sent the wrong data
• 401 Unauthorized: Authentication needed
• 403 Forbidden: Authorization needed
• 404 Not found: Resource or entity doesn’t exist
• 405 Method Not Allowed: The action that you’re trying to execute isn’t allowed
• 500 Internal Server Error: Something went wrong on the server side

There are other status codes, but they aren’t as widely used as the ones mentioned here.

Once you know the status code and the data that you want to return, which can be done in the request body, you must serialize that to JSON and communicate to the client that you’re using JSON format by using the Content-Type header.

So, now that we know we need a URI, an HTTP method, an HTTP request body, an HTTP status code, an HTTP response body, and a couple of Content-Type headers, let’s see some examples of this using curl.

Let’s imagine we have an API with users resources in the http://localhost:8888/api/v1/users URI, and we want to create a new entity for that. We know the resource URI, and because we want to create a new entity, we know that we have to use the POST verb on the resource URI and pass new user information in the request’s body in JSON format. So, we can do something like this:

$ curl -X POST -d'{"username": "john.doe", "name": "John", "surename": "Doe"}' -H "Content-Type: application/json" http://localhost:8888/api/v1/users

If something isn’t OK in the request, we could get something such as a 400 Bad Request error from our API, but if everything goes well, we expect to receive a 201 Create status code and an output similar to the following:

{
    "id": 42,
    "username": "jonh.doe",
    "name": "John",
    "surname": "Doe",
}

Here, we can see how the HTTP protocol conveys all the information that we need in both directions to decide what to do, how to do it, and what the final result of the operations will be. But that’s only part of the power that comes with the fact that REST is usually built on top of HTTP. Let’s take a look at some of its other advantages.

Other HTTP features used in REST

HTTP is a very rich protocol that provides many different HTTP features that we, as API developers, can leverage to improve our APIs.

Probably the most widely used one is the use of query parameters to convey extra information, such as pagination or filtering. For example, if I don’t want to get all the users in the system, I could use something such as /api/v1/users?page=0&perPage=100, providing parameters to the users resource to get only page number 0 and only 100 entities per page. The parameters that are used and their effects are defined by the REST API developer, so there’s no standard way of doing this. However, common patterns are used frequently in REST services.

Another important HTTP feature that’s widely used is cache headers. HTTP provides a set of HTTP headers related to the cache that instruct the server or the client on how we want the cache on the other side to behave. This can imply substantial performance improvements. For example, I can use the Cache-Control: max-age=60 HTTP header to instruct the client to cache this request’s result and use it for 60 seconds. That would help me avoid requesting the same data again during the next 60 seconds if there are more petitions.

The HTTP protocol provides security, authorization, and authentication mechanisms. CORS and the Authorization header are good examples. HTTP also provides security-related headers, something we’re going to explore more deeply in Chapter 7.

Custom headers are something that can be seen in the REST API. It’s another way to send or receive information between the programs that are communicating. Custom headers are prefixed with X- to differentiate them from the official headers. For example, I could deliver pagination information by adding headers such as X-Page: 0 and X-Entities-Per-Page: 100, and perhaps the server could reply with the entities and another header, such as X-Has-Next-Page: true. This kind of approach isn’t especially common or widely used, but it’s good to know that there’s that possibility.

The last important aspect that has made REST so successful is the fact that it uses a “human-readable” data representation. A huge percentage of APIs use JSON as their serialization mechanism, allowing the API consumers to inspect and debug the data they get from the API service quickly. This “easy-to-read” data provides developers with a boost in understanding the API and/or the errors that occur due to bad usage or service implementation errors.

So, REST defines resources and entities and is built on top of HTTP. But what does all this have to do with Go, and why should you use Go and not any other language? Let’s see.

First, there are very good reasons to use Go for API development, and they’re the same for other languages. I’m heavily biased because I love the Go programming language for multiple reasons. Still, I’m going to explain why I think Go is a great language for API development and why I think it’s ahead of other languages.

One of the main reasons is the language itself. If you know Go, you already know how simple, elegant, and fast it can be. Having a language that keeps your problem straightforward is ideal for REST API development. Go tends to provide low-complexity solutions and avoid boilerplate code. APIs are often created as small functions that give access to your logic. A language that gets straight to the point and allows you to write that code succinctly and directly is a blessing in this context.

Another good reason why you should use Go for REST is its extensive standard library, which provides you with most of the building blocks you’ll need to build an API without any external dependencies. JSON encoding/decoding, HTTP servers, URL routing, query parameters parsing, and header handling are all built into the standard library. When you start writing your API in Go, you already have all that you need to build the API. You may need some other libraries, depending on the project, but even if that’s required, the Go ecosystem surrounding APIs is, in general, very mature.

Apart from the standard library, the language itself provides particular advantages when you’re working with programs that operate concurrently, such as an API where you have multiple clients consuming that API at the same time. In those scenarios, its built-in concurrency primitives and lightweight processes (goroutines) make your applications ultra-performant out of the box, almost without you needing to think about it. The built-in HTTP server handles the requests in independent goroutines, so your API will be running concurrently, even if you don’t use goroutines anywhere in your code.

There are other popular language options out there for API development, from Python to Rust to others such as C# and Java. But, from my experience, Go falls in the sweet spot between performance, a high degree of control, development speed, and robustness. Of course, I’m biased because Go is my language of choice, but I’ve worked with other languages, and I haven’t found something in the same sweet spot that Go has for building APIs.

In conclusion, you can build APIs with any language, but not all of them provide the same value as Go. While we explore this in this book, I will show you how good Go is at writing APIs and how far you can go using the standard library almost exclusively.

As mentioned in the Technical requirements section, you’ll need the Go compiler to build our examples. If you haven’t done so yet, you can learn how to install it at http://go.dev.

Let’s start building a very simple API from scratch in Go. We aren’t going to dive too deep into the details here, but I’m going to explain what’s going on in general terms. We’ll explore the specifics in the following chapters. For now, we only want to get a very simple API up and running.

Creating the project

The first step is to create our project. For that, we’re going to need to create a directory and initialize our project package there:

$ mkdir myapi
$ cd myapi
$ go mod init myapi

In the first and second lines, we’re creating the new directory and entering it.

In the third line, when we run go mod init myapi, we’re defining the root import path of our project. I’m using myapi here, but it’s more common to use something such as github.com/username/reponame as the import path as it refers to the path of the repository in GitHub, or any other service that you use to host your code.

Once you’ve initialized the Go project, we can start writing the first version of our program. For now, we’re going to build a “Hello World” program as an HTTP server in a file called main.go.

Writing a “Hello World” program

In this section, we’re going to write one of the simplest APIs that we can build: a Hello World API that allows us to make requests and always returns Hello World as a string. Let’s see the code:

package main
import (
    "fmt"
    "net/http"
)
func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, "Hello World")
    })
    http.ListenAndServe(":8888", nil)
}

If you’re familiar with Go, then most of the code here will be very straightforward. You only need to pay a bit more attention to http.HandleFunc, which registers a new available URL in our HTTP server. This tells our server how to handle requests on that URL using the handler defined inline there.

To handle these requests, we can just write the response handler alongside the result that we want to transmit. Don’t worry too much about the details here – we’ll dive deeper into this in the following chapters.

Once the handler has been defined and bound to a URL, we need to start listening on a port and run the web server. We can do this using the http.ListenAndServe function.

With that, we already have an HTTP server that gets requests and gives responses. Now, let’s execute the server from the command line.

Running your server

Your API server runs just like every other program in Go, so you have two options. The first option is to run the program directly by executing the following command:

$ go run .

Alternatively, you can build your program and run the final executable:

$ go build .
$ ./main

In both cases, you’ll end up with a process running that exposes port 8888 and accepts HTTP requests. But you don’t need to believe me; let’s check it out.

Accessing the API

The most straightforward approach that you can follow to check whether your server is running and behaving as expected is to open your web browser and go to http://localhost:8888. If everything went well, you should see the text Hello World, as shown in Figure 1.1.

Figure 1.4 – Hello World in a browser

Another option is to use curl, a command-line-based HTTP client, to get this same information:

$ curl http://localhost:8888
Hello World

However, there are other ways to access our API apart from using HTTP clients such as a web browser or curl. You can also build your API client. For that, you’ll still need to use HTTP under the hood, but you’ll access the API using code, not other applications.

Creating the client project

We need to create a project for the client, similar to what we did for the server. To do so, we can execute similar commands:

$ mkdir myclient
$ cd myclient
$ go mod init myclient

With that, our project is ready and we can start working on the code.

Writing the API client

An API client is nothing other than the code that accesses the API. In our case, we’ll access the API directly using an HTTP request and print the response to the terminal:

package main
import (
    "fmt"
    "io"
    "net/http"
)
func main() {
    result, err := http.Get("http://localhost:8888/")
    if err != nil {
        panic(err)
    }
    defer result.Body.Close()
    data, err := io.ReadAll(result.Body)
    if err != nil {
        panic(err)
    }
    fmt.Println("Response:", string(data))
}

Here, we use the http.Get function to make a GET request to our API server. Our server will generate Hello World and send that as a response to our request.

So, we read the body that’s sent by the server and use that data to print the necessary information to the console. Easy, right?

Running the client

Before running the client, be sure that your server is up and running. If not, the client will fail because it won’t be able to connect to the server. Again, you can run the client however you prefer. I prefer using the go run option because it’s just one command:

$ go run .
Response: Hello World

With that, we receive the response from the server and print it to the terminal.

Hooray! You’ve created your first API server and client in Go and communicated two independent programs through that new API. But this is just the beginning.

In this chapter, we conveyed a lot of general information about APIs in general – specifically REST APIs. We talked about other alternatives, such as gRPC, GraphQL, and SOAP before diving deeper into the principles and concepts that are used in REST and how they’re used in the HTTP protocol.

Regarding REST, we took a closer look at how REST leverages HTTP verbs, status codes, headers, and request/response bodies to communicate the data, format, and actions required to provide us with a solid foundation to build our APIs.

Finally, we learned how to build a very simple API server and client and how to communicate information between them using the Go standard library using the HTTP package.

In the next chapter, we’ll explore the different options and tools that we have to better understand existing REST APIs.

Unlock this book’s exclusive benefits now Scan this QR code or go to packtpub.com/unlock, then search this book by name.
Note: Keep your purchase invoice ready before you start.