Use DTOs to Enforce a Layer of Indirection | Node.js w/ TypeScript

Last updated Feb 24th, 2021
DTOs help you create a more stable RESTful API; they protect your API clients from changes made on the server.

Have you ever heard the expression that "all problems in computer science can be solved by adding another layer of indirection"?

I'm not sure about all, but if the structural problems in software design can be reduced to being either an issue with coupling or cohesion, then yeah - this makes a lot of sense.

Indirection

To understand the concept of indirection, we need to understand the fact that software doesn't do a whole lot unless we can connect things together. This means that components (whether it be a class, object, package, server, or client application) typically rely on other components to do something meaningful.

Take web applications for example. Most web applications implement the client-server architecture. This usually means that the client relies on the server.

Client relies on server

And if we're being realistic, it's a lot more likely that there will be several clients that rely on the server (like a web, mobile, and desktop client). Notion is a great example of this. They have a web, mobile, and desktop app.

Clients rely on server

In this dependency relationship, the clients rely on the server. A lot. It means that if, for some reason, the server were to change things or break, that change or breakage has the potential to ripple into each client that relies on it.

This is the nature of tight coupling. It puts the clients in an awkward position of needing to constantly keep on top of the latest changes to the server and protect itself against failure. A substantial change could break the entire client. This isn't great. Not great at all.

In fact, just this week, I realized that the browsers introduced a new security feature called strict-origin-when-cross-origin that was stopping solidbook.io readers from getting into the online Wiki. Since my application depends on the browser, it means I'll always have to keep on top of things the browser does that could cause my app to break.

What can we do about this problem?

Introduce a layer of indirection

Let me walk you through a real-world example of how to use DTOs — Data-Transfer Objects to introduce stability and protect clients from change in a RESTful API.

A trivial RESTful API controller

Consider this very stripped down, simple controller that pulls the current user and returns a raw Sequelize ORM object as a result of the GetCurrentUser RESTful API call.

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res) => {
    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ where: { user_id: userId } });
    return user;
  }
}

For a long time, this is how I used to build out my controllers in my RESTful API layer.

We're not doing anything fancy. We're just using an ORM to get the data we need and return it to the caller. Straightforward, right?

What's the shape of the user object on our RESTful API? Because it's hooked up directly to our ORM, it'll return whatever that, the ORM, returns.

{
  firstName: 'Khalil',
  lastName: 'Stemmler',
  id: 1,
  email: `khalil@khalilstemmler.com`,
  isAdmin: true,
  isStudent: false,
  isEmployer: false,
}

Cool. That's alright. The design could be better. But let's imagine that this is what we've chosen and it's been like that for quite some time now. Consider that client teams have used our API and built things on top of it already.

Now this happens.

We want to change the API

After months of living with our design, we realize that we don't like it — particularly the isAdmin, isStudent, isEmployer part.

We realize that we're moving towards having more and more roles and that hard-coding the roles as a part of the user entity actually breaks domain encapsulation. The user entity has become somewhat coupled the specifics of other subdomains (students, jobs).

An astute software designer may also recognize that this is a violation of the Open-Closed principle. A module should be open for extension but closed for modification, right? But it seems that every time we want to add a new role (ie: extend this entity's use case), we have to modify it directly (by adding a new column to the database in the form of an is[role] structure). Not good.

We goofed up here. That's OK. The design got us to this point, and that's what matters. We're here today and we want to refactor it.

We decide that it would make more sense to have something like this instead:

{
  firstName: 'Khalil',
  lastName: 'Stemmler',
  id: 1,
  email: `khalil@khalilstemmler.com`,
  roles: ['Admin']
}

That's much better. The concept of roles seems much more appropriate within a users subdomain. Not only that, but we're doing good by the Open-Closed Principle: we can extend this entity by adding a role, not changing the structure of the entity.

Lovely.

So, we go ahead and make the change by writing a Sequelize migration, looking through our code and changing every controller that mentions a user to do the following.

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res) => {
    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ 
			where: { user_id: userId },
			include: [				{ model: sequelize.db.Role, as: 'Roles' },			]		});

    return {      ...user,      // This is how we maintain the structure      isAdmin: user.Roles.find((r) => r.name === "Admin"),      isStudent: user.Roles.find((r) => r.name === "Student"),      isEmployer: user.Roles.find((r) => r.name === "Employer")    }  }

Hopefully we've found and changed every place in our codebase where we return a user; because if we miss one, we might break the API for our clients.

Disadvantages to this approach

Not only is this messy, but even despite best efforts, it's too easy to break. Clients are practically hooked up directly to any changes we make to our database. There is almost no encapsulation from the implementation details.

Dec

The main issues here are that:

  • Our API is not stable
  • It forces us to get the design absolutely correct the first time around (pretty unlikely)
  • We'll have to scour our codebase for other places affected by database changes
  • It makes implementing API changes incredibly hard
  • Drives us to tolerate bad design decisions because changing them is just too much of a hassle
  • And we know where this leads — it's the broken windows theory. Software rot is just down the road.

Enforcing a layer of indirection

Using Data Transfer Objects

DTOs are data transfer objects. They're not much. They're simple interfaces or classes that represent the shape of your API.

The idea is to use DTOs to create a contract for your API.

If we modelled the initial shape of the user entity as a part of our RESTful API, we'd likely have a DTO that looked like this.

interface UserDTO {
  firstName: String;
  lastName: String
  id: Number;
  email: String;
  isAdmin: boolean;
  isStudent: boolean;
  isEmployer: boolean;
}

Then we'd use another pattern, the data mapper pattern, to hold the responsibility of mapping the raw sequelize object to the correct DTO shape

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res): Promise<UserDTO> => {    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ 
			where: { user_id: userId },
			include: [
				{ model: sequelize.db.Role, as: 'Roles' },
			]
		});
    return UserMap.toDTO(user);  }
}

This can be improved even further by encapsulating the ORM logic in a repository.

const GetCurrentUserController = (userRepo: IUserRepo) => {
  return async (req: Express.req, res: Express.res): Promise<UserDTO> => {    const userId = req.decoded.userId;
    const user = await userRepo.findUserByUserId(userId);
    return UserMap.toDTO(user);
  }
}

Advantages to this approach

Now instead of relying on the server (and everything the server relies on), the clients rely only on the DTOs.

Relying on DTOs

The advantages are that:

  • The DTOs become a strict contract for our API. The clients rely on it. The server implements it.
  • We've implemented architectural dependency inversion between the client and the server.
  • The DTOs act as a layer of indirection and shield the clients from internal changes to the way the API is resolved.
  • Data mappers act as a single location for object transformations. Instead of going through the codebase and changing API code that deals with raw sequelize users all over the place, it's done in one place.

If this smells a lot like the things GraphQL does, it's because it is. GraphQL acts as a strictly-typed abstraction that clients rely on and services implement.

Different objects for different needs

I think a layered architecture is one of the best things you can do for non-trivial web applications of complexity.

You have domain objects (that your domain layer needs), data transfer objects (that your clients need), and the raw sequelize objects (that your database needs). The mapper handles transformations between all of 'em.

Continue reading

Want to learn how to do implement this in your next project?

Read the next post here: Implementing DTOs, Mappers & the Repository Pattern using the Sequelize ORM [with Examples] - DDD w/ TypeScript



Discussion

Liked this? Sing it loud and proud 👨‍🎤.


16 Comments

Commenting has been disabled for now. To ask questions and discuss this post, join the community.

Kasun
2 years ago

Hi, can you do a few tutorials about functional programming patterns in the enterprise?

Khalil Stemmler
2 years ago

Yes! It's on my list. I'll bump it up. Thanks, Kasun.

Wagner
2 years ago

Hi, what do you think of implementing the DTO also on the client to protect changes in the api?

Khalil Stemmler
2 years ago

I'd say this is a must for this to work well. Both the client and the server rely on the DTOs.


When you're using a GraphQL client library like Apollo Client, you get type-safety right out of the box because the client relies on the GraphQL API (essentially the DTOs) and it can immediately tell you when your queries are broken.


We want the same behavior in a RESTful context. If we're using TypeScript, we can get that same feeling of instant feedback if we import our DTOs and use them to strictly type API responses.

Yury P.
2 years ago

That's good. How do you propose to organize the interaction of backend and frontend by DTO if they are two different applications (nuxt.js and nest.js, for example)?

Khalil Stemmler
2 years ago

Great question. This is going to largely depend on how your project is organized.


1. Monorepo (both client and server in same repository)

  • Probably the easiest scenario. I recommend writing a script to copy the DTOs from your server code to a `dtos` folder in your client app every time you start the app in dev mode, build it, or run the tests.
  • I'd start here if possible.

2. Different client and server repos

  • Publish an entirely separate client-side library for the client to rely on. This library contains the DTOs. You could also make this library robust, encapsulating all of the data-fetching logic using Axios, Fetch, or whatever HTTP lib you prefer. This is a great approach if you have several client applications. The client API needs to change in a single place. This should make it easier for other teams to keep an up-to-date API. The downside is that you're maintaining another repository and all the challenges of doing so comes with this approach.

Donny Roufs
2 years ago

Hey! First of I bought your book and it really is a great source to get myself to the next step of becoming a well rounded developer! I have a couple questions some of them are out of the scope of this article tho!


1. Where do we store our timestamps for the given entity? Is it okay to give the domain entity for example a createdAt property? I would assume it is since the repository's responsibility is to map it back to a domain entity before giving it back to the service right?


2. Where does the dto mapping happen? The way I understand the flow is: http controller does some kind of presence validation on the request, maps it to a requestdto and gives it to the service. The service maps this to a domain entity and passes it to the repository. Repository does its thing and maps it back to a domain entity, service maps it to a reponse dto and gives it to the controller. However I see an issue here;


what if the request had additional data for the query. Lets imagine a count, we want X amount entities back. Do we pass this as a seperate argument to the repository? Or does this mean that we do not map the request dto to the domain entity when we pass it to repo? But that means that we cant do our business validation.


Again thanks for all the hard work!

MY
2 years ago

Amazing article for obviously many developers but especially for those like me lacking basic information and experience from the robust principles of the good api / app architecture.

Łukasz
2 years ago

Hi Khalil! Thanks for writing this up, this topic has been itching my brain for few years (literally!).

The problem I have with DTOs is that they introduce a lot of boilerplate, which makes them sometimes infeasible to use. Just for your simple example you need:

  • UserDto on the client side
  • UserDto on the server side
  • User domain object
  • [Optional] UserRecord object (for SQL)
  • And a bunch of mappers between all of those.


Moreover, what happens if you need your RESTful resource to be returning a DTO which combines two other DTOs/Entities, say: UserWithAddress,a nd you only need a handful of properties/fields from each class (ex. firstName, lastName from User; city: from Address)? How and where do you construct them?


I can't find a silver bullet here for neat API design that would scale up from small to large projects.


Thanks!


P.S. A great fan of your work.

Jayakrishnan
2 years ago

Using sequalize is efficient you think? Since I have done benchmark many times and they are pretty slow for converting JSON to raw query. Also when we join multiple tables sequalize is using not doing it inefficient way. And sometimes transactions are also a bit weird.

KELVIN T PETER
2 years ago

This is a great article, keep doing the good work

mert
2 years ago

So the "domain object" is just the e.g. UserModel?

jhanvi mehta
a year ago

Thank you for this amazing information. I am working on the node.js for my project But I beginner in this field so not more knowledge about that but I am in your blog and I read it and that was so informative for me. Also right now many companies provide a node.js development service.

Gaurav Sharma
a year ago

Data Transfer Object is an important part to transfer data from point a to point b, I must say your article has all information, and It clears all the doubts who want to know about how the indirection layer is applied through DTOs.


Thank you for the article which helps many nodejs development companies.

adam
a year ago

I'm finishing my college education and I can say that the only right decision in the process of studying was to buy research papers and not spend time writing on my own. The main thing in this is to find really qualified writers from an online service, which I did.

Jean dos santos
a year ago

I didn't understand well the use of the DTO yet.

dianacrown85
10 months ago

The greater part of individuals are back of building muscles in numerous ways and I have been anticipating that it should make through the most various ways. As indicated by how to compose a school exposition at exploratory writing research paper that do my number theory homework, it's is proposed to get them normally through exercises instead of enhancements.

Bienfait
9 months ago

I understand most of the part except dataMappers. I dont know to integrate it in my code !

Ruben Perez
7 months ago

Amazing explanation!!

Regards!


Stay in touch!



About the author

Khalil Stemmler,
Software Essentialist ⚡

I'm Khalil. I turn code-first developers into confident crafters without having to buy, read & digest hundreds of complex programming books. Using Software Essentialism, my philosophy of software design, I coach developers through boredom, impostor syndrome, and a lack of direction to master software design and architecture. Mastery though, is not the end goal. It is merely a step towards your Inward Pull.



View more in Enterprise Node + TypeScript



You may also enjoy...

A few more related articles

Functional Error Handling with Express.js and DDD | Enterprise Node.js + TypeScript
How to expressively represent (database, validation and unexpected) errors as domain concepts using functional programming concept...
Flexible Error Handling w/ the Result Class | Enterprise Node.js + TypeScript
Purposefully throwing errors can have several negative side effects to the readability and traceability of your code. In this arti...
Clean & Consistent Express.js Controllers | Enterprise Node.js + TypeScript
In this article, we explore how to structure a clean and consistent Express.js controller by using abstraction and encapsulation w...
Better Software Design with Application Layer Use Cases | Enterprise Node.js + TypeScript
In this article, we explore how organizing application logic as Use Cases in the application layer helps to make large typescript ...

Want to be notified when new content comes out?

Join 15000+ other Software Essentialists learning how to master The Essentials of software design and architecture.

Get updates