This article is part of the series gRPC With Go Crash Course
In this part of the course, you will implement slightly more challenging RPC routines which involve utilising the gRPC streaming capabilities.
These are essentially a way to achieve client-side or server-side pagination. However, in contrast to normal HTTP APIs, in gRPC these streaming utilities are first-class citizens rather than a useful add-on (e.g. via web sockets).
There is also support for bidirectional streams, which we will explore in the follow-up exercise. In this one, we’ll focus on unidirectional (client-side/server-side) streaming RPCs.
The Starting Point
If you completed the previous exercise, you can continue from where you left off.
To get started, run this command to clone the necessary exercise materials in a convenient folder:
git clone --branch v3-simple-rpc https://github.com/preslavmihaylov/go-grpc-crash-course
NOTE: If you are using GOPATH mode, you should clone the repo exactly in <your-gopath>/src/github.com/preslavmihaylov/go-grpc-crash-course.
Next up, install the libraries you’ll need by executing this at the heart of the repo:
go mod downloadThis line will install the required dependencies in your $GOPATH.
Your Goal
In this exercise, we’ll be exploring client-side and server-side unidirectional streams.
Client-side stream example:
Server-side stream example:
First, take a look around the scaffold and familiarize yourself with what you’re building.
You have three main folders – casino, client, payment_statements
- casino – this contains the source code for a casino microservice. This will contain the bulk of the backend logic.
- client – this is a client application, which connects to the casino service and allows the user to use the application via a CLI interface.
- payment_statements – this contains the source code for a payment statements microservice. It will be solely responsible for generating a user’s payment statement given a list of payments. The casino service will connect to this one and create user payment statements upon request.
All the service schema are in the idl directory.
Finally, you have make_protos.sh, which is a simple bash script which compiles all your protobuf schemas together.
This is a utility script, enabling you to more easily re-compile proto schemas, instead of having to dig in your bash history for the correct protoc command.
The IDLs & initial service integrations are already setup for you. Additionally, all the basic RPC routines (BuyTokens, Withdraw & TokenBalance) are already implemented.
These were done in the previous exercises.
What we have left is to implement all streaming RPCs in our services. For all the locations where you have to chime in, you have a TODO: Implement comment or panic in there.
Take a look around and see where all those locations are.
In this exercise, we’ll be focusing on implementing unidirectional stream RPCs. These are native gRPC mechanisms to achieve pagination of either output or input.
If you want to attempt the challenge on your own, read on. Otherwise skip to the Full Walkthrough section.
In particular, what you have to complete is:
In casino/main.go:
- GetPayments – this routine takes in a User input and returns a stream of payments. This is a server-side streaming API.
What you have to do, is return all of the user’s payments, which are stored in casinoServer.userToPayments. They should be bundled in a commonpb.Payment struct.
Hint: use the Send function on casinopb.Casino_GetPaymentsServer to sequentially send payments through the stream - GetPaymentStatement – this routine creates a payment statement, which is a user-friendly statement of a user’s payment history & current balance.
The routine is a simple request-response RPC. However, in order to create the payment statement, you have to utilize the payment_statements service.
In particular, you have to forward all the user’s payments to the payment_statements’s CreateStatement routine. The way you forward them is by using client-side streams.
After the payment statement is created, you should return the resulting data (a string type) to the user, bundled in a commonpb.PaymentStatement struct.
Hint: Create a new client stream with paymentStatementsClient.CreateStatement(ctx) and use the Send function on the returned struct to sequentially send payments to the payment_statements service
Hint: After you’re done, use the stream’s CloseAndRecv function to receive the final result after all payments have been sent to payment_statements.
In payment_statements/main.go:
- CreateStatement – This routine takes in a stream of commonpb.Payment and bundles it into a payment statement the user can view.
Leverage the toPaymentStatement function which takes in an array of payments and returns the payments statement corresponding to them.
This function has already been created for you.
Hint: use Send from payment_statements.PaymentStatements_CreateStatementServer to sequentially receive the payments from the casino web service. Finally, use SendAndClose to close the stream and return the resulting payment statement.
In client/commands.go:
- payments – invoke the GetPayments routine from casino using the gRPC client instance of it – available in global variable client).
The UserID is available in a global variable username.
Collect the payments in a []*commonpb.Payment array.
If there was an error, return it. If the call was successful, return a message: Here’s your payment history:\n{paymentHistory}.
To nicely format the payments into a payment history, use the paymentHistoryString function, which has been already created for you.
- paymentStatement – invoke the GetPaymentStatement routine from casino.
Return the resulting data (in commonpb.PaymentStatement.Data) as-is.
Those are all the requirements you have to accomplish as part of this exercise.
If everything is completed well, this is what the final result should look like after you run all the binaries:
In the following section, you’ll find a complete walkthrough of the exercise. Use it in case you get stuck!
Good luck!
For help, refer to the gRPC Go Tutorial.
Full Walkthrough
Remember! Before going through this walkthrough, make sure you’ve attempted to complete the exercise on your own.
Refer to it if you get stuck or are interested in how I did it.
No cheating! 🙈🙈🙈
Now that we’ve gotten that out of the way, let’s get started!
Step 1. Implement GetPayments in casino
We’ll start by implementing the GetPayments routine on the server-side.
First, open casino/main.go and add some debug logging & helper variables:
Next comes the interesting part. We’ll return all the payments to the user. However, we’re not just going to return them as an array. We’ll stream them one by one using stream.Send.
We’ll also add some error-handling and return a nil error in the end to indicate all went well:
You might think that using a stream for our use-case is an overkill. We could just return an array to make matters simpler and you’d be right!
However, imagine you have millions of payments. You can’t just return them all at once. Instead, you would have to employ some kind of pagination.
But rather than dealing with e.g. page markers & constant request-responses to get all the payments you need on the client-side, gRPC offers you a way to stream all the payments in a single TCP session. That’s where this is actually very useful and efficient!
Step 2. Invoke GetPayments on the client-side
The next step is to invoke the newly created routine on our client-side.
Go to client/commands.go and initialize the GetPayments stream using the casino client:
Next, process payments one by one as they come from the stream and note that whenever you receive an error io.EOF, then that means the stream is closed and there is no more data.
At that point, you should return the payments you collected in a nice format leveraging the paymentHistoryString function:
Now, test that your payment history is correctly shown in the CLI:
Step 3. Implement CreateStatement in payment_statements
For the first time, we’ll diverge from the client and casino folders and go to payment_statements/main.go to implement CreateStatement.
This is a client-streaming routine, meaning that the client invoking this routine will stream input data to us. In the end, after we’ve processed the input, we should return a result.
Handling the client-side stream is very similar to what we did in the previous step.
The only thing we’re doing different this time is the error-handling to demonstrate a different way of handling errors in stream-processing situations.
In addition to that, this time we have to use stream.SendAndClose to return the result:
Step 4. Implement GetPaymentStatement in casino & client
The final step from this exercise is to implement the GetPaymentStatement routine in casino and invoke it in the client.
Let’s start from casino/main.go and create a new stream for the CreateStatement routine we implemented just now.
We’ll leverage the globally available paymentStatementsClient for that purpose:
Next, we’ll start forwarding the payments to the payments_statements service for it to aggregate them into a statement:
Finally, we’ll close the stream and return the resulting payment statement:
Now, let’s invoke that newly created routine in client/commands.go:
Finally, test that everything works well and voila!
Finale
Congratulations. 👏👏👏
You’ve successfully completed the exercise.
You should now have a good understanding of how to use & create streaming-based RPC routines.
However, we’re yet to cover the more complex bidirectional streams gRPC has to offer.
Don’t worry, we’ll explore that in the next exercise, so stay tuned. 🙌🙌🙌
If you want to view the final solution for this exercise, checkout the v4-unidirectional-streams branch.
