Use DTOs to Enforce a Layer of Indirection | Node.js w/ TypeScript

Last updated Feb 24th, 2021
DTOs help you create a more stable RESTful API; they protect your API clients from changes made on the server.

Have you ever heard the expression that "all problems in computer science can be solved by adding another layer of indirection"?

I'm not sure about all, but if the structural problems in software design can be reduced to being either an issue with coupling or cohesion, then yeah - this makes a lot of sense.

Indirection

To understand the concept of indirection, we need to understand the fact that software doesn't do a whole lot unless we can connect things together. This means that components (whether it be a class, object, package, server, or client application) typically rely on other components to do something meaningful.

Take web applications for example. Most web applications implement the client-server architecture. This usually means that the client relies on the server.

Client relies on server

And if we're being realistic, it's a lot more likely that there will be several clients that rely on the server (like a web, mobile, and desktop client). Notion is a great example of this. They have a web, mobile, and desktop app.

Clients rely on server

In this dependency relationship, the clients rely on the server. A lot. It means that if, for some reason, the server were to change things or break, that change or breakage has the potential to ripple into each client that relies on it.

This is the nature of tight coupling. It puts the clients in an awkward position of needing to constantly keep on top of the latest changes to the server and protect itself against failure. A substantial change could break the entire client. This isn't great. Not great at all.

In fact, just this week, I realized that the browsers introduced a new security feature called strict-origin-when-cross-origin that was stopping solidbook.io readers from getting into the online Wiki. Since my application depends on the browser, it means I'll always have to keep on top of things the browser does that could cause my app to break.

What can we do about this problem?

Introduce a layer of indirection

Let me walk you through a real-world example of how to use DTOs — Data-Transfer Objects to introduce stability and protect clients from change in a RESTful API.

A trivial RESTful API controller

Consider this very stripped down, simple controller that pulls the current user and returns a raw Sequelize ORM object as a result of the GetCurrentUser RESTful API call.

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res) => {
    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ where: { user_id: userId } });
    return user;
  }
}

For a long time, this is how I used to build out my controllers in my RESTful API layer.

We're not doing anything fancy. We're just using an ORM to get the data we need and return it to the caller. Straightforward, right?

What's the shape of the user object on our RESTful API? Because it's hooked up directly to our ORM, it'll return whatever that, the ORM, returns.

{
  firstName: 'Khalil',
  lastName: 'Stemmler',
  id: 1,
  email: `khalil@khalilstemmler.com`,
  isAdmin: true,
  isStudent: false,
  isEmployer: false,
}

Cool. That's alright. The design could be better. But let's imagine that this is what we've chosen and it's been like that for quite some time now. Consider that client teams have used our API and built things on top of it already.

Now this happens.

We want to change the API

After months of living with our design, we realize that we don't like it — particularly the isAdmin, isStudent, isEmployer part.

We realize that we're moving towards having more and more roles and that hard-coding the roles as a part of the user entity actually breaks domain encapsulation. The user entity has become somewhat coupled the specifics of other subdomains (students, jobs).

An astute software designer may also recognize that this is a violation of the Open-Closed principle. A module should be open for extension but closed for modification, right? But it seems that every time we want to add a new role (ie: extend this entity's use case), we have to modify it directly (by adding a new column to the database in the form of an is[role] structure). Not good.

We goofed up here. That's OK. The design got us to this point, and that's what matters. We're here today and we want to refactor it.

We decide that it would make more sense to have something like this instead:

{
  firstName: 'Khalil',
  lastName: 'Stemmler',
  id: 1,
  email: `khalil@khalilstemmler.com`,
  roles: ['Admin']
}

That's much better. The concept of roles seems much more appropriate within a users subdomain. Not only that, but we're doing good by the Open-Closed Principle: we can extend this entity by adding a role, not changing the structure of the entity.

Lovely.

So, we go ahead and make the change by writing a Sequelize migration, looking through our code and changing every controller that mentions a user to do the following.

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res) => {
    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ 
			where: { user_id: userId },
			include: [				{ model: sequelize.db.Role, as: 'Roles' },			]		});

    return {      ...user,      // This is how we maintain the structure      isAdmin: user.Roles.find((r) => r.name === "Admin"),      isStudent: user.Roles.find((r) => r.name === "Student"),      isEmployer: user.Roles.find((r) => r.name === "Employer")    }  }

Hopefully we've found and changed every place in our codebase where we return a user; because if we miss one, we might break the API for our clients.

Disadvantages to this approach

Not only is this messy, but even despite best efforts, it's too easy to break. Clients are practically hooked up directly to any changes we make to our database. There is almost no encapsulation from the implementation details.

Dec

The main issues here are that:

  • Our API is not stable
  • It forces us to get the design absolutely correct the first time around (pretty unlikely)
  • We'll have to scour our codebase for other places affected by database changes
  • It makes implementing API changes incredibly hard
  • Drives us to tolerate bad design decisions because changing them is just too much of a hassle
  • And we know where this leads — it's the broken windows theory. Software rot is just down the road.

Enforcing a layer of indirection

Using Data Transfer Objects

DTOs are data transfer objects. They're not much. They're simple interfaces or classes that represent the shape of your API.

The idea is to use DTOs to create a contract for your API.

If we modelled the initial shape of the user entity as a part of our RESTful API, we'd likely have a DTO that looked like this.

interface UserDTO {
  firstName: String;
  lastName: String
  id: Number;
  email: String;
  isAdmin: boolean;
  isStudent: boolean;
  isEmployer: boolean;
}

Then we'd use another pattern, the data mapper pattern, to hold the responsibility of mapping the raw sequelize object to the correct DTO shape

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res): Promise<UserDTO> => {    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ 
			where: { user_id: userId },
			include: [
				{ model: sequelize.db.Role, as: 'Roles' },
			]
		});
    return UserMap.toDTO(user);  }
}

This can be improved even further by encapsulating the ORM logic in a repository.

const GetCurrentUserController = (userRepo: IUserRepo) => {
  return async (req: Express.req, res: Express.res): Promise<UserDTO> => {    const userId = req.decoded.userId;
    const user = await userRepo.findUserByUserId(userId);
    return UserMap.toDTO(user);
  }
}

Advantages to this approach

Now instead of relying on the server (and everything the server relies on), the clients rely only on the DTOs.

Relying on DTOs

The advantages are that:

  • The DTOs become a strict contract for our API. The clients rely on it. The server implements it.
  • We've implemented architectural dependency inversion between the client and the server.
  • The DTOs act as a layer of indirection and shield the clients from internal changes to the way the API is resolved.
  • Data mappers act as a single location for object transformations. Instead of going through the codebase and changing API code that deals with raw sequelize users all over the place, it's done in one place.

If this smells a lot like the things GraphQL does, it's because it is. GraphQL acts as a strictly-typed abstraction that clients rely on and services implement.

Different objects for different needs

I think a layered architecture is one of the best things you can do for non-trivial web applications of complexity.

You have domain objects (that your domain layer needs), data transfer objects (that your clients need), and the raw sequelize objects (that your database needs). The mapper handles transformations between all of 'em.

Continue reading

Want to learn how to do implement this in your next project?

Read the next post here: Implementing DTOs, Mappers & the Repository Pattern using the Sequelize ORM [with Examples] - DDD w/ TypeScript



Discussion

Liked this? Sing it loud and proud 👨‍🎤.


9 Comments

Submit
Kasun
9 months ago

Hi, can you do a few tutorials about functional programming patterns in the enterprise?

Khalil Stemmler
9 months ago

Yes! It's on my list. I'll bump it up. Thanks, Kasun.

Wagner
9 months ago

Hi, what do you think of implementing the DTO also on the client to protect changes in the api?

Khalil Stemmler
9 months ago

I'd say this is a must for this to work well. Both the client and the server rely on the DTOs.


When you're using a GraphQL client library like Apollo Client, you get type-safety right out of the box because the client relies on the GraphQL API (essentially the DTOs) and it can immediately tell you when your queries are broken.


We want the same behavior in a RESTful context. If we're using TypeScript, we can get that same feeling of instant feedback if we import our DTOs and use them to strictly type API responses.

Yury P.
9 months ago

That's good. How do you propose to organize the interaction of backend and frontend by DTO if they are two different applications (nuxt.js and nest.js, for example)?

Khalil Stemmler
9 months ago

Great question. This is going to largely depend on how your project is organized.


1. Monorepo (both client and server in same repository)

  • Probably the easiest scenario. I recommend writing a script to copy the DTOs from your server code to a `dtos` folder in your client app every time you start the app in dev mode, build it, or run the tests.
  • I'd start here if possible.

2. Different client and server repos

  • Publish an entirely separate client-side library for the client to rely on. This library contains the DTOs. You could also make this library robust, encapsulating all of the data-fetching logic using Axios, Fetch, or whatever HTTP lib you prefer. This is a great approach if you have several client applications. The client API needs to change in a single place. This should make it easier for other teams to keep an up-to-date API. The downside is that you're maintaining another repository and all the challenges of doing so comes with this approach.

Donny Roufs
9 months ago

Hey! First of I bought your book and it really is a great source to get myself to the next step of becoming a well rounded developer! I have a couple questions some of them are out of the scope of this article tho!


1. Where do we store our timestamps for the given entity? Is it okay to give the domain entity for example a createdAt property? I would assume it is since the repository's responsibility is to map it back to a domain entity before giving it back to the service right?


2. Where does the dto mapping happen? The way I understand the flow is: http controller does some kind of presence validation on the request, maps it to a requestdto and gives it to the service. The service maps this to a domain entity and passes it to the repository. Repository does its thing and maps it back to a domain entity, service maps it to a reponse dto and gives it to the controller. However I see an issue here;


what if the request had additional data for the query. Lets imagine a count, we want X amount entities back. Do we pass this as a seperate argument to the repository? Or does this mean that we do not map the request dto to the domain entity when we pass it to repo? But that means that we cant do our business validation.


Again thanks for all the hard work!

MY
8 months ago

Amazing article for obviously many developers but especially for those like me lacking basic information and experience from the robust principles of the good api / app architecture.

Łukasz
8 months ago

Hi Khalil! Thanks for writing this up, this topic has been itching my brain for few years (literally!).

The problem I have with DTOs is that they introduce a lot of boilerplate, which makes them sometimes infeasible to use. Just for your simple example you need:

  • UserDto on the client side
  • UserDto on the server side
  • User domain object
  • [Optional] UserRecord object (for SQL)
  • And a bunch of mappers between all of those.


Moreover, what happens if you need your RESTful resource to be returning a DTO which combines two other DTOs/Entities, say: UserWithAddress,a nd you only need a handful of properties/fields from each class (ex. firstName, lastName from User; city: from Address)? How and where do you construct them?


I can't find a silver bullet here for neat API design that would scale up from small to large projects.


Thanks!


P.S. A great fan of your work.

Jayakrishnan
6 months ago

Using sequalize is efficient you think? Since I have done benchmark many times and they are pretty slow for converting JSON to raw query. Also when we join multiple tables sequalize is using not doing it inefficient way. And sometimes transactions are also a bit weird.

KELVIN T PETER
3 months ago

This is a great article, keep doing the good work

mert
a month ago

So the "domain object" is just the e.g. UserModel?


Stay in touch!



About the author

Khalil Stemmler,
Developer Advocate @ Apollo GraphQL ⚡

Khalil is a software developer, writer, and musician. He frequently publishes articles about Domain-Driven Design, software design and Advanced TypeScript & Node.js best practices for large-scale applications.



View more in Enterprise Node + TypeScript



You may also enjoy...

A few more related articles

Functional Error Handling with Express.js and DDD | Enterprise Node.js + TypeScript
How to expressively represent (database, validation and unexpected) errors as domain concepts using functional programming concept...
Flexible Error Handling w/ the Result Class | Enterprise Node.js + TypeScript
Purposefully throwing errors can have several negative side effects to the readability and traceability of your code. In this arti...
Clean & Consistent Express.js Controllers | Enterprise Node.js + TypeScript
In this article, we explore how to structure a clean and consistent Express.js controller by using abstraction and encapsulation w...
Better Software Design with Application Layer Use Cases | Enterprise Node.js + TypeScript
In this article, we explore how organizing application logic as Use Cases in the application layer helps to make large typescript ...

Want to be notified when new content comes out?

Join 10000+ other developers learning about Domain-Driven Design and Enterprise Node.js.

I won't spam ya. 🖖 Unsubscribe anytime.

Get updates