MongoDB

3. MongoDB

We use MongoDB to store user account information and files

3.1 Mongoose

We use Mongoose for modeling the data, is an object modeling tool for MongoDB. The reference is here

3.1.1 Preload

To preload the model, firstly import the configuration. Also import the mongoose npm module

File location: loaders\mongoDB.js

const config = require('../config');
const {sleep} = require('../utils')

const mongoose = require('mongoose');

mongoose.connect(config.mongodb.addr, {
  useNewUrlParser: true,
  useUnifiedTopology: true,
  useFindAndModify: false,
  useCreateIndex: true,
});

Then, connect to MongoDB and add an error handler. Also, we need to get log from console to make sure MongoDB is connected.

mongoose.connect(config.mongodb.addr, {
  useNewUrlParser: true,
  useUnifiedTopology: true,
  useFindAndModify: false,
  useCreateIndex: true,
});

const db = mongoose.connection;

db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function () {
  // we're connected!
  console.log('MongoDB connected.')

});

3.1.2 Modeling

To create a mongoDB model by Mongoose, you can do the following:

First, in a class file of /model folder, load the mooge NPM package

const mongoose = require('mongoose');
const Schema = mongoose.Schema;

Create a schema:

const UserSchema = new Schema({
  username: {type: String, unique: true, required: true},
  email: String,
}, {timestamps: {createdAt: 'created_at', updatedAt: 'updated_at'}});

Model the schema:

const MongoDBUserModel = mongoose.model("User", UserSchema, "User");

Then you can insert the data in the database by following the schema:

const mongoDBModel = new MongoDBUserModel({
            username: "exampleuser",
            email: "test@example.com",
          })
await mongoDBModel.save()

If you want to see other detail you need, you can view the reference here.

3.1.3 Queries

Mongoose provides functions to find, update and delete data, the following are examples

Find documents in a collection

const users = await MongoDBUserModel.find({})

Find one specific by giving a condition filter

const user = await MongoDBUserModel.findOne({username: "exampleuser",
                                             email: "test@example.com"})

Update data by giving a condition filter

await MongoDBUserModel.findOneAndUpdate({username: "exampleuser"}, {email: "test2@example.com"})

Delete data by giving a condition filter

const user = await MongoDBUserModel.deleteOne({username: "exampleuser",
                                             email: "test@example.com"})

If you want to see other details you need, you can view the reference here.

3.2 Grid-FS

Grid-FS is a file storage tool for MongoDB, the reference is here. For Mongoose, we use Mongoose GridFS.

3.2.1 Setting and preload

In the mongodb loader file \loaders\mongoDB.js, on top of the original loader setting, import the NPM package

const {createModel} = require('mongoose-gridfs');

Load the attachment

let Attachment;
db.once('open', function () {
  ...

  // initialize GridFS
  Attachment = createModel({
    modelName: 'Attachment',
    connection: db
  });
});

Implement a asynchronous function and load the attachment

async function getAttachment() {
  while (!Attachment) {
    await sleep(100);
  }
  return Attachment;
}

module.exports = {getAttachment};

After that, we can implement the function to be able to upload and download file from MongoDB.

3.2.2 Upload file

The following is the implementation of upload file, you can read details here. Currently, it is only used to store resources for opportunity listings.

const Attachment = await getAttachment();
    let closedByServer;

    const writeStream = Attachment.bucket.writeFile({filename, contentType}, readStream);

    return new Promise((resolve, reject) => {

      // Only store the record when the actual file is stored in MongoDB.
      writeStream.on('finish', async (file) => {
        const fileData = new MongoDBFileModel({fileId: file._id});
        await fileData.save();
        closedByServer = true;
        resolve(file._id);
      });

      writeStream.on('error', reject);

      // if the client or server aborted the request
      readStream.on('close', () => {
        // If the client aborted the connection, abort the write steam.
        // This will also cleanup the data already wrote to MongoDB.
        if (!closedByServer)
          writeStream.abort();
      });

      readStream.on('error', reject);
    })

After the file is uploaded, it will be given file ID to help with getting the file later on. To save the file ID, we can create different mongoDB model to store the file ID.

const MongoDBFileSchema = Schema({
  fileId: {type: String, unique: true},
  uploadDate: {type: Date, default: Date.now()}
});

const MongoDBFileModel = mongoose.model('MongoDBFile', MongoDBFileSchema, 'MongoDBFile')

3.2.3 Get file

The following is the implementation of download file, you can read details here

async function getFile(req, res, next) {
    const {fileId} = req.params;
    
    const Attachment = await getAttachment()
    try {
      const file = await Attachment.findById(fileId);
      const readStream = Attachment.bucket.readFile({_id: ObjectId(fileId)})
    
      res.set('Content-Type', file.contentType);
      readStream.pipe(res).on('error', function (err) {
        res.json({error: err})
      })
    } catch (e) {
      next(e);
    }
}

3.3 MongoDB Compass

MongoDB Compass provides a user-friendly UI to visualize the MongoDB, you can view the detail here.

3. MongoDB#

3.1 Mongoose#

3.1.1 Preload#

3.1.2 Modeling#

3.1.3 Queries#

3.2 Grid-FS#

3.2.1 Setting and preload#

3.2.2 Upload file#

3.2.3 Get file#

3.3 MongoDB Compass#