Zero To Production #0: Foreword

Zero To Productionis a book that I will be writing in the open, publishing one chapter at a time on this blog. If you’d like to be notified when an episode comes out you can subscribe to the mailing list:

Who Is This Book For

The initial response to a random tweet about the idea of spinning up this project has been overwhelming.

Hundreds of subscribers to a rushed email newsletter.

Several emails and private messages detailing this or that specific setup and the challenges they are currently facing.

I have written and re-written the table of contents more than a couple of times trying to zero in on what I want to cover. I realised by the second version of it that I can’t satisfy many of those emails and DMs.

The Rust ecosystem has had a remarkable focus on smashing adoption barriers with amazing material geared towards beginners and newcomers, a relentless effort that goes from documentation to the continuous polishing of the compiler diagnostics.

There is value in serving the largest possible audience.

At the same time, trying to always speak to everybody can have harmful side-effects: material that would be relevant to intermediate and advanced users but definitely too much too soon for beginners ends up being neglected.

I struggled with it first-hand when I started to play around with async/await .

There was a significant gap between the knowledge I needed to be productive and the knowledge I had built reading The Rust Book or working in the Rust numerical ecosystem.

I wanted to get an answer to a straight-forward question:

Can Rust be a productive language for API development?

Yes.

But it can take some time to figure out how .

That’s why I am writing this book.

I am writing this book for the seasoned backend developers who have read The Rust Book and are now trying to port over a couple of simple systems.

I am writing this book for the new engineers on my team, a trail to help them make sense of the codebases they will contribute to over the coming weeks and months.

I am writing this book for a niche whose needs I believe are currently underserved by the articles and resources available in the Rust ecosystem.

I am writing this book for myself a year ago.

To socialise the knowledge gained during the journey: what does your toolbox look like if you are using Rust for backend development in 2020? What are the design patterns? Where are the pitfalls?

If you do not fit this description but you are working towards it I will do my best to help you on the journey: while we won’t be covering a lot of material directly (e.g. most Rust language features) I will try to provide references and links where needed to help you pick up/brush off those concepts along the way.

Let’s get started.

What Is This Book About

The world of backend development is vast .

The socio-technical context you operate into has a huge impact on the optimal tools and practices to tackle the problem you are working on.

For example, trunk-based development works extremely well to write software that is continuously deployed in a Cloud environment.

The very same approach might fit poorly the business model and the challenges faced by a team that sells software that is hosted and run on-premise by their customers - they are more likely to benefit from a Gitflow approach.

If you are working alone, you can just push straight to master.

There are few absolutes in the field of software development and I feel it’s beneficial to clarify your point of view when evaluating the pros and cons of any technique or approach.

Zero To Production will focus on the challenges of writing Cloud-native applications in a team of four or five engineers with different levels of experience and proficiency.

Cloud-native applications

Defining what Cloud-native application means is by itself enough of a topic for a whole book on its own. Instead of prescribing what Cloud-native applications should look like , we can lay down what we expect them to do .

Paraphrasing Cornelia Davis, we expect Cloud-native applications:

• To achieve high-availability while running in fault-prone environments;
• To allow us to continuously release new versions with zero downtime;
• To handle dynamic workloads (e.g. request volumes).

These requirements have a deep impact on the viable solution space for the architecture of our software.

High availability implies that our application should be able to serve requests with no downtime even if one or more of our machines suddenly starts failing (a common occurrence in a Cloud environment). This forces our application to be distributed - there should be multiple instances of it running on multiple machines.

The same is true if we want to be able to handle dynamic workloads - we should be able to measure if our system is under load and throw more compute at the problem by spinning up new instances of the application. This also requires our infrastructure to be elastic to avoid overprovisioning and its associated costs.

Running a replicated application influences our approach to data persistence - we will avoid using the local filesystem as our primary storage solution, relying instead on databases for our persistence needs.

Zero To Production will thus extensively cover topics that might seem tangential to pure backend application development. But Cloud-native software is all about rainbows and DevOps, therefore we will be spending plenty of time on topics traditionally associated with the craft of operating systems.

We will cover how to instrument your Rust application to collect logs, traces and metrics to be able to observe our system.

We will cover how to set up and evolve your database schema via migrations.

We will cover all the material required to use Rust to tackle both day one and day two concerns of a Cloud-native API.

Working in a team

The impact of those three requirements goes beyond the technical characteristics of our system: it influences how we build our software.

To be able to quickly release a new version of our application to our users we need to be sure that our application works.

If you are working on a solo project you can rely on your thorough understanding of the whole system: you wrote it, it might be small enough to fit entirely in your head at any point in time.

If you are working in a team on a commercial project, you will be very often working on code that was neither written or reviewed by you. The original authors might even not be around anymore.

You will end up being paralysed by fear every time you are about to introduce changes if you are relying on your comprehensive understanding of what the code does to prevent it from breaking.

You want automated tests.

Running on every commit. On every branch. Keeping master healthy.

You want to leverage the type system to make undesirable states difficult or impossible to represent.

You want to use every tool at your disposal to empower each member of the team to evolve that piece of software. To contribute fully to the development process even if they might not be as experienced as you or equally familiar with the codebase or the technologies you are using.

Zero To Production will therefore put a strong emphasis on test-driven development and continuous integration from the get-go - we will have a CI pipeline set up before we even have a web server up and running!

We will be covering techniques such as black-box testing for APIs and HTTP mocking - not wildly popular or well documented in the Rust community yet extremely powerful.

We will also borrow terminology and techniques from the Domain Driven Design world, combining them with type-driven design to ensure the correctness of our systems.

Our main focus is enterprise software : correct code which is expressive enough to model the domain and supple enough to support its evolution over time.

We will thus have a bias for boring and correct solutions, even if they incur a performance overhead that could be optimised away with a more careful and chiseled approach.

Get it to run first, optimise it later (if needed).

Table of Contents

The Table of Contents is provisional and might change over time. The draft below is the most accurate picture at this point in time.

• Who Is This Book For
• What Is This Book About

1. Getting Started
   
   • Installing The Rust Toolchain
   • IDE Setup
   • Project Setup
   • Continuous Integration
2. Our Driving Example
   
   • What Should Our Newsletter Do?
   • Working In Iterations
3. Sign Up A New Subscriber
   
   • Application Scaffolding
   • Our First Integration Test
   • Reading Request Data
   • Adding A Database
   • Persisting A New Subscriber
4. Publish A Newsletter Issue
   
   • Writing A REST Client
   • Mocking Third-Party APIs
5. Logging
   
   • The Facade Pattern
   • Logging Levels
   • Log Setup
6. Reject Invalid Subscribers
   
   • Result
   • Modeling With Types #1
7. Survive Delivery Failures
   
   • Simulating API Errors
8. Tracing
   
   • Structured Logging
   • Spans
   • OpenTelemetry
   • Jaeger
9. Send A Confirmation Email On Sign Up
   
   • Migrating Your Database
   • Modeling With Types #2
   • Handling Confirmations
   • Send Newsletter Only To Confirmed Subscribers
10. Metrics
    
    • Prometheus
    • Grafana
11. Send Emails Asynchronously
    
    • Adding A Message Broker
    • Enqueueing Tasks
12. Fulfilling Email Tasks
    
    • Adding An Actor Queue Worker
    • Basic Retries
    • Failure Injection
    • Idempotency
13. Benchmarking
    
    • Cargo bench
    • Criterion
    • Load testing

Zero To Productionis a book that I will be writing in the open, publishing one chapter at a time on this blog. If you’d like to be notified when an episode comes out you can subscribe to the mailing list: