« Previous Tutorial Next Tutorial »

Today we’re gonna mess with our DB in various ways. Don’t worry, it’s not going to be that big of a deal. It’s going to involve deleting one of our existing collections, though. Before we get there, we need to do some work on our add album API route. This is a big set of changes, because now when a user adds an album, we’re also going to save not only that album, but that album’s artists to our local database. This means fewer requests to the Discogs API, because we won’t need to contact it when users are viewing their list (or others’ lists).

So let’s get started with two easy changes. First, open /models/album.js and replace line 7, which has an incorrectly spelled key (way to go, me!) with the following:

  discogsId: { type: Number, unique: true },

The unique thing is important. We’re setting discogsId as a unique index, which means the database will throw an error if we try to submit two albums with the same id. We’re going to run a check for this anyway, but it’s still a useful backup to have.

Save this file, and then create a new file in /models called artist.js. We’re going to save a subset of the artist data that Discogs provides to our database, just like with albums. Here’s the code:

const mongoose = require('mongoose');

const Schema = mongoose.Schema;

const Artist = new Schema({
  discogsId: { type: Number, unique: true },
  images: [Schema.Types.Mixed],
  members: [Schema.Types.Mixed],
  name: String,
  namevariations: [String],
  profile: String,
  urls: [String],
});

module.exports = mongoose.model('Artist', Artist);

I know nameVariations isn’t camel-cased correctly but it’s coming in from the API like that, and I didn’t want to add a bunch of extra code (or an auto-camel-casing module) just to fix it. So … avert your eyes!

Save this file, and head for /routes/api/albums.js. This is where the major changes come in, and there’s a lot of them. I spent a bunch of time writing and rewriting this code, trying to get something that worked reliably with async / await and didn’t have to do a ton of looping and nesting functions. Took a while, but I got there! I’ve created a bunch of helper functions to help keep things clean, and we’re wrapping all of our Discogs calls in promises to better facilitate awaiting them.

Let’s get started by importing our models. At the very top of the file, add these two lines:

const Album = require('../../models/album.js');
const Artist = require('../../models/artist.js');

Now we need two helper functions. The first checks if an album exists in our local database already and, if it doesn’t, adds it. Below line 19, add a padding line and then the following code:

// Check if album exists and if not, save it
const saveAlbum = async (albumInfo) => {
  let errors = false;
  const albumQuery = await Album.findOne({ discogsId: albumInfo.id });
  if (!albumQuery) {
    const albumInfoModified = Object.assign({ discogsId: albumInfo.id }, albumInfo);
    const newAlbum = new Album(albumInfoModified);
    await newAlbum.save((error) => {
      if (error) { errors = true; }
    });
  }
  if (errors) {
    return false;
  }
  return true;
};

Note that we’re adding our discogsId field during the Object.assign step. This is important because otherwise the first album we ever add will have a “null” value for that field, and since it’s unique, no further albums can be added, since they’ll also have null as a value, and Mongoose will go “nah, there’s already one in there with that value” and reject the addition.

Onward to our next helper function. This does the same thing for artists, but it’s a little more complex because it iterates through an array rather than only taking a single artist at a time. Here’s the code:

// Check each artist in an array to see if it exists and if not, save it
const saveArtists = async (artists) => {
  const formattedArtists = artists.map((artist) => {
    const newArtist = Object.assign({ discogsId: artist.id }, artist);
    return newArtist;
  });
  try {
    const result = await Artist.insertMany(
      formattedArtists,
      { ordered: false },
      (error) => {
        if (error && error.code !== 11000) {
          return false;
        }
        return true;
      },
    );
    return result;
  } catch (error) {
    if (error.code !== 11000) {
      return false;
    }
    return true;
  }
};

We’re using MongoDB’s built-in insertMany method, here, but there’s a small problem with that. By default, the method craps out if any part of your array matches an existing record, and stops adding any additional artists who are listed. So if you’re adding a collaboration album between Kendrick Lamar and, uh … Twenty One Pilots, or something, and you already have Kendrick in your database, Twenty One Pilots won’t get added due to Mongoose throwing an error.

BUT WAIT! We can fix that, sort of, with that {ordered: false} line. This tells the database to keep accepting further entries even if it rejects one or more of them as already existing. This is fantastic, and would be even more fantastic if Mongo and Mongoose didn’t still throw an error. Because they do, we have to wrap it in a try/catch block to keep script execution running, and we need to check for error code eleven thousand, because that’s the error we want to ignore.

OK, moving on into the actual POST, which if your spacing matches mine, should now start on line 64. The following dozen lines remain the same, but we need to add some internal helpers here, too. Below line 76, add a padding line, and then these two functions:

  // Get a single artist from Discogs
  const discogsGetArtist = artistId => new Promise((resolve) => {
    discogsDB.getArtist(artistId, (err, data) => {
      resolve(data);
    });
  });

  // Loop through artists and hit the Discogs API for each
  const getArtists = async (artists) => {
    const results = artists.map((artist) => {
      const artistInfo = discogsGetArtist(artist.id);
      return artistInfo;
    });
    return Promise.all(results);
  };

We avoid using await in a loop, here, because it’s bad practice. Basically, if you do that, the loop hangs until the response comes back, and then continues moving. Instead, we use Promise.all to tell our code that we want to make an asynchronous hit for each artist, and when they’ve all returned, pass along the final results. This wouldn’t work if we hadn’t wrapped our Discogs call in a promise, but we did!

Now we need to add some additional steps to our try / catch block. Below line 100 (which is a padding line), add this code:

    // Save it to the MusicList DB if it's not already there
    const albumSaved = await saveAlbum(albumInfo);
    if (!albumSaved) { return JSON.stringify(new Error('There was a problem saving the album to the database.')); }

    // Go through the album's artists and get their full info from Discogs
    const artistsInfo = await getArtists(albumInfo.artists);

    // Save the artists to the MusicList DB if they're not already there
    const artistsSaved = await saveArtists(artistsInfo);
    if (!artistsSaved) { return JSON.stringify(new Error('There was a problem saving the artist to the database.')); }

As you can see, we’re awaiting several results here and then doing stuff once we have it. We’re also in a few cases erroring out if things break. The hope is that they won’t, of course, but we’d rather not have them just failing in the background with no notice.

We’re set here. The rest of the code is the same. Save this file. Now hop over to a terminal window and fire up your MongoDB client by typing mongo. We’re going to delete our existing albums collection entirely, so type the following:

use musiclist

Followed by:

db.albums.drop()

We need to do this because we initialized the collection already, but with the wrong settings for discogsId. It may not strictly be necessary, but it’s more of a precautionary measure.

Now let’s blank out any albums we previously added to our test user. The easiest way to do this is the following line, except replacing YOURUSERNAME with, well, your username.

db.users.update({ username: "YOURUSERNAME" }, {$set: { albums: [], artists: [] }})

Once you’ve run that, we should be all set. Here’s how we’re going to test this: leave your mongo console open for a bit and switch to a browser. Hard-refresh your page for safety, and then log in and head for /albums. Do a search on an artist you like who has at least two albums. Then add at least two albums to your list by clicking the buttons. As in the previous tutorial, you’ll see the “Already listed” text appear, which is great.

Now head back to your mongo console, and type the following:

db.artists.find().length()

This should give you a return value of one, because we’ve added two albums, but only one artist. If you’re getting a return value of more than one, it may be that one of the albums you added had multiple artists. To check and make sure, you can do:

db.artists.find()

And look over the resulting output. It’s ugly, but you should be able to tell whether you’re getting duplicate artists or not.

Once you’ve confirmed there are no dupes, do this:

db.albums.find().length()

This will output two (or more if you added more albums).

Congrats, you’re saving albums and artists to your local database, and you’re keeping dupes out. That’s good stuff.

In the next tutorial we’re going to allow users to search for and add artists. It’s going to be a lot of code re-use, which means we should be able to crank through it really quickly. See you then!

« Previous Tutorial Next Tutorial »