TypeScript Express tutorial #13. Using Mongoose virtuals to populate documents

• 1. TypeScript Express tutorial #1. Middleware, routing, and controllers
• 2. TypeScript Express tutorial #2. MongoDB, models and environment variables
• 3. TypeScript Express tutorial #3. Error handling and validating incoming data
• 4. TypeScript Express tutorial #4. Registering users and authenticating with JWT
• 5. TypeScript Express tutorial #5. MongoDB relationships between documents
• 6. TypeScript Express tutorial #6. Basic data processing with MongoDB aggregation
• 7. TypeScript Express tutorial #7. Relational databases with Postgres and TypeORM
• 8. TypeScript Express tutorial #8. Types of relationships with Postgres and TypeORM
• 9. TypeScript Express tutorial #9. The basics of migrations using TypeORM and Postgres
• 10. TypeScript Express tutorial #10. Testing Express applications
• 11. TypeScript Express tutorial #11. Node.js Two-Factor Authentication
• 12. TypeScript Express tutorial #12. Creating a CI/CD pipeline with Travis and Heroku
• 13. TypeScript Express tutorial #13. Using Mongoose virtuals to populate documents

Throughout this series, we’ve created some schemas and models for documents. We’ve also established ways to define relationships between them. We sometimes find ourselves needing many different properties in our documents. They sometimes overlap each other, and we might want to avoid duplicating the data in the database. Also, we might want to avoid two-way referencing that we discussed in the fifth part of this series . The solution to all of the above issues might be  virtuals . To understand them, we also inspect  getters and  setters first.

You can find all of the code from this series in this repository . Feel free to give it a star and share it.

Getters and Setters in Mongoose

The first concept to wrap our heads around is the idea of getters and  setters . They allow us to execute custom logic whenever we are getting and setting properties in a document.

Getters

With getters, we can modify the data of a document. Let’s assume that the user has the credit card number that we don’t want to fully present.

const userSchema = new mongoose.Schema(
  {
    email: String,
    name: String,
    password: String,
    creditCardNumber: {
      type: String,
      get: (creditCardNumber: string) => {
        return `****-****-****-${
          creditCardNumber.substr(creditCardNumber.length - 4)
        }`;
      },
    },
  },
  {
    toJSON: { getters: true },
  },
);

In the above example, every time the User document is pulled from the database, we obfuscate the credit card number to protect it.

To make our changes reflect in the response, we need to pass additional options to the mongoose . Schema constructor. When we respond with the data of a user, Express calls the toJSON function internally. If we want the getters to apply, we need to set  getters : true as above.

Documents also have the toObject method that we can also customize similarly.

A useful use-case for the above might be to make sure that the password of the user never gets sent back in response.

const userSchema = new mongoose.Schema(
  {
    email: String,
    name: String,
    password: {
      type: String,
      get: (): undefined => undefined,
    },
  },
  {
    toJSON: {
      getters: true,
    },
  },
);

By creating such a getter, we avoid leaking sensitive data. We just need to remember that if we need to access the password, we need the Document.prototype.get() function with an additional option.

const isPasswordMatching = await bcrypt.compare(
  logInData.password,
  user.get('password', null, { getters: false }),
);

Since we rarely need to access the password property, it is a more fitting approach to hide it by default.

Setters

Using setters, we can modify the data before it is populated in the database. With them, we can inject additional logic and modify the data.

import createPasswordHash from './createPasswordHash'
 
const userSchema = new mongoose.Schema({
  email: String,
  name: String,
  password: {
    type: String,
    set: (plaintextPassword: string) => {
      return createPasswordHash(plaintextPassword);
    },
  },
});

You might prefer to have the above logic inside your services instead of using setters. Even if that’s the case, it is good to be aware of the above functionality.

Mongoose virtuals

A Virtual is an additional property of a document. We can get it and  set it, but it does not persist in the MongoDB database.

A typical example might be with names. First, let’s create a firstName property without the use of virtuals.

const userSchema = new mongoose.Schema({
  email: String,
  firstName: String,
  lastName: String,
  fullName: String,
  password: String,
});

​The above approach has a few issues. By persisting the fullName data into MongoDB, we duplicate the information, which is unnecessary. Also, imagine that at some point, we would like to support a middle name and make it a part of the  fullName . A more fitting approach would be to attach the  fullName dynamically.

Getters

We can do the above with the use of virtuals . Let’s create it along with a  getter .

const userSchema = new mongoose.Schema(
  {
    email: String,
    firstName: String,
    lastName: String,
    password: String,
  },
  {
    toJSON: { virtuals: true },
  },
);
 
userSchema.virtual('fullName').get(function () {
  return `${this.firstName} ${this.lastName}`;
});

Now, every time we pull the user entity from the database, we also create the fullName property dynamically.

Similarly to regular getters, we also need to pass additional options to the Document.prototype.get() constructor. Thanks to adding  toJSON : { virtuals : true } above, our virtual properties are visible when converting to JSON.

Now, when we inspect our database, we can see that the fullName field is not populated:

Setters

Virtuals also support setters . With them, we can set multiple properties at once.

userSchema.virtual('fullName')
  .get(function () {
    return `${this.firstName} ${this.lastName}`;
  })
  .set(function (fullName: string) {
    const [firstName, lastName] = fullName.split(' ');
    this.set({ firstName, lastName });
  });

In the above example, we set the firstName and  lastName properties based on the  fullName . Within the  setter function of a Virtual,  this refers to the document.

Populating virtuals

One of the most useful features of virtuals is populating documents from another collection.

In the fifth part of this series , we discuss the relationships between documents. The direction of the reference is an essential mention in the above article. Let’s bring one of the examples again:

const postSchema = new mongoose.Schema({
  author: {
    ref: 'User',
    type: mongoose.Schema.Types.ObjectId,
  },
  content: String,
  title: String,
});

In the above schema, we keep the reference to users in the Post document. Therefore, the documents look like that:

{
  "author": "5e40382687aa217496466bad",
  "title": "Lorem ipsum",
  "content": "Dolor sit amet"
}

Once we settle for one of the above approaches to storing references, we don’t have the information on the other side of the relationship. If we store the reference inside of a post, the user document does not hold the information about posts:

{
  "_id": "5e40382687aa217496466bad",
  "name": "John",
  "email": "john@smith.com"
}

We could do it the other way around instead.

const userSchema = new mongoose.Schema({
  email: String,
  name: String,
  password: String,
  posts: [
    {
      ref: 'Post',
      type: mongoose.Schema.Types.ObjectId,
    },
  ],
});

Doing so would mean that the posts do not contain information about the authors. We could implement two-way referencing, as described in the mentioned article, but it comes with some disadvantages.

Solving the issue with virtuals

To tackle the above problem, we can implement a virtual property:

userSchema.virtual('posts', {
  ref: 'Post',
  localField: '_id',
  foreignField: 'author',
});

In the above code, we connect the _id of a user to the  author of a post. We also tell Mongoose through ref wich model to populate documents from.

The only thing that’s left is to use the populate function.

src/user/user.controller.ts

private initializeRoutes() {
  this.router.get(`${this.path}/:id`, authMiddleware, this.getUserById);
}

private getUserById = async (request: Request, response: Response, next: NextFunction) => {
  const id = request.params.id;
  const user = await this.user.findById(id).populate('posts');
  if (user) {
    response.send(user);
  } else {
    next(new UserNotFoundException(id));
  }
}

You might want to implement some additional authorization to prevent every authenticated user to have the access to all the users

An important issue that we should address here is that the populate function takes some additional time to finish. A way to tackle this issue is to implement an additional query param.

src/user/user.controller.ts

private getUserById = async (request: Request, response: Response, next: NextFunction) => {
  const id = request.params.id;
  const userQuery = this.user.findById(id);
  if (request.query.withPosts === 'true') {
    userQuery.populate('posts').exec();
  }
  const user = await userQuery;
  if (user) {
    response.send(user);
  } else {
    next(new UserNotFoundException(id));
  }
}

Now we populate the posts array only if it was explicitly asked for through query parameters by calling  users / [ id ] ? withPosts = true .

Summary

In this article, we’ve gone through the idea of virtuals. To fully grasp the concept, we’ve investigated getters and  setters first. Thanks to doing so, we’ve managed to improve our code. By populating the virtual properties, we’ve found another solution to a common issue with references. Doing all of the above gives us lots of new possibilities on how to tackle everyday challenges.