Use DTOs to Enforce a Layer of Indirection | Node.js w/ TypeScript

Have you ever heard the expression that "all problems in computer science can be solved by adding another layer of indirection"?

I'm not sure about all, but if the structural problems in software design can be reduced to being either an issue with coupling or cohesion, then yeah - this makes a lot of sense.

Indirection

To understand the concept of indirection, we need to understand the fact that software doesn't do a whole lot unless we can connect things together. This means that components (whether it be a class, object, package, server, or client application) typically rely on other components to do something meaningful.

Take web applications for example. Most web applications implement the client-server architecture. This usually means that the client relies on the server.

And if we're being realistic, it's a lot more likely that there will be several clients that rely on the server (like a web, mobile, and desktop client). Notion is a great example of this. They have a web, mobile, and desktop app.

In this dependency relationship, the clients rely on the server. A lot. It means that if, for some reason, the server were to change things or break, that change or breakage has the potential to ripple into each client that relies on it.

This is the nature of tight coupling. It puts the clients in an awkward position of needing to constantly keep on top of the latest changes to the server and protect itself against failure. A substantial change could break the entire client. This isn't great. Not great at all.

In fact, just this week, I realized that the browsers introduced a new security feature called strict-origin-when-cross-origin that was stopping solidbook.io readers from getting into the online Wiki. Since my application depends on the browser, it means I'll always have to keep on top of things the browser does that could cause my app to break.

What can we do about this problem?

Introduce a layer of indirection

Let me walk you through a real-world example of how to use DTOs — Data-Transfer Objects to introduce stability and protect clients from change in a RESTful API.

A trivial RESTful API controller

Consider this very stripped down, simple controller that pulls the current user and returns a raw Sequelize ORM object as a result of the GetCurrentUser RESTful API call.

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res) => {
    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ where: { user_id: userId } });
    return user;
  }
}

For a long time, this is how I used to build out my controllers in my RESTful API layer.

We're not doing anything fancy. We're just using an ORM to get the data we need and return it to the caller. Straightforward, right?

What's the shape of the user object on our RESTful API? Because it's hooked up directly to our ORM, it'll return whatever that, the ORM, returns.

{
  firstName: 'Khalil',
  lastName: 'Stemmler',
  id: 1,
  email: `khalil@khalilstemmler.com`,
  isAdmin: true,
  isStudent: false,
  isEmployer: false,
}

Cool. That's alright. The design could be better. But let's imagine that this is what we've chosen and it's been like that for quite some time now. Consider that client teams have used our API and built things on top of it already.

Now this happens.

We want to change the API

After months of living with our design, we realize that we don't like it — particularly the isAdmin, isStudent, isEmployer part.

We realize that we're moving towards having more and more roles and that hard-coding the roles as a part of the user entity actually breaks domain encapsulation. The user entity has become somewhat coupled the specifics of other subdomains (students, jobs).

An astute software designer may also recognize that this is a violation of the Open-Closed principle. A module should be open for extension but closed for modification, right? But it seems that every time we want to add a new role (ie: extend this entity's use case), we have to modify it directly (by adding a new column to the database in the form of an is[role] structure). Not good.

We goofed up here. That's OK. The design got us to this point, and that's what matters. We're here today and we want to refactor it.

We decide that it would make more sense to have something like this instead:

{
  firstName: 'Khalil',
  lastName: 'Stemmler',
  id: 1,
  email: `khalil@khalilstemmler.com`,
  roles: ['Admin']
}

That's much better. The concept of roles seems much more appropriate within a users subdomain. Not only that, but we're doing good by the Open-Closed Principle: we can extend this entity by adding a role, not changing the structure of the entity.

Lovely.

So, we go ahead and make the change by writing a Sequelize migration, looking through our code and changing every controller that mentions a user to do the following.

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res) => {
    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ 
			where: { user_id: userId },
			include: [				{ model: sequelize.db.Role, as: 'Roles' },			]		});

    return {      ...user,      // This is how we maintain the structure      isAdmin: user.Roles.find((r) => r.name === "Admin"),      isStudent: user.Roles.find((r) => r.name === "Student"),      isEmployer: user.Roles.find((r) => r.name === "Employer")    }  }

Hopefully we've found and changed every place in our codebase where we return a user; because if we miss one, we might break the API for our clients.

Disadvantages to this approach

Not only is this messy, but even despite best efforts, it's too easy to break. Clients are practically hooked up directly to any changes we make to our database. There is almost no encapsulation from the implementation details.

The main issues here are that:

• Our API is not stable
• It forces us to get the design absolutely correct the first time around (pretty unlikely)
• We'll have to scour our codebase for other places affected by database changes
• It makes implementing API changes incredibly hard
• Drives us to tolerate bad design decisions because changing them is just too much of a hassle
• And we know where this leads — it's the broken windows theory. Software rot is just down the road.

Enforcing a layer of indirection

Using Data Transfer Objects

DTOs are data transfer objects. They're not much. They're simple interfaces or classes that represent the shape of your API.

The idea is to use DTOs to create a contract for your API.

If we modelled the initial shape of the user entity as a part of our RESTful API, we'd likely have a DTO that looked like this.

interface UserDTO {
  firstName: String;
  lastName: String
  id: Number;
  email: String;
  isAdmin: boolean;
  isStudent: boolean;
  isEmployer: boolean;
}

Then we'd use another pattern, the data mapper pattern, to hold the responsibility of mapping the raw sequelize object to the correct DTO shape

const GetCurrentUserController = (sequelize) => {
  return async (req: Express.req, res: Express.res): Promise<UserDTO> => {    const userId = req.decoded.userId;
    const user = await sequelize.db.User.findById({ 
			where: { user_id: userId },
			include: [
				{ model: sequelize.db.Role, as: 'Roles' },
			]
		});
    return UserMap.toDTO(user);  }
}

This can be improved even further by encapsulating the ORM logic in a repository.

const GetCurrentUserController = (userRepo: IUserRepo) => {
  return async (req: Express.req, res: Express.res): Promise<UserDTO> => {    const userId = req.decoded.userId;
    const user = await userRepo.findUserByUserId(userId);
    return UserMap.toDTO(user);
  }
}

Advantages to this approach

Now instead of relying on the server (and everything the server relies on), the clients rely only on the DTOs.

The advantages are that:

• The DTOs become a strict contract for our API. The clients rely on it. The server implements it.
• We've implemented architectural dependency inversion between the client and the server.
• The DTOs act as a layer of indirection and shield the clients from internal changes to the way the API is resolved.
• Data mappers act as a single location for object transformations. Instead of going through the codebase and changing API code that deals with raw sequelize users all over the place, it's done in one place.

If this smells a lot like the things GraphQL does, it's because it is. GraphQL acts as a strictly-typed abstraction that clients rely on and services implement.

Different objects for different needs

I think a layered architecture is one of the best things you can do for non-trivial web applications of complexity.

You have domain objects (that your domain layer needs), data transfer objects (that your clients need), and the raw sequelize objects (that your database needs). The mapper handles transformations between all of 'em.

Continue reading

Want to learn how to do implement this in your next project?

Read the next post here: Implementing DTOs, Mappers & the Repository Pattern using the Sequelize ORM [with Examples] - DDD w/ TypeScript