How to Create an API with Pagination, Filtering, and Sorting Using Node.js and Express

APIs are essential components of modern web applications. They enable different software systems to communicate with each other, often involving fetching and manipulating data. When building APIs, it's important to provide clients with flexible options to retrieve data efficiently. This includes pagination, filtering, and sorting.

In this blog post, I will walk you through how to create an API using Node.js and Express that supports pagination, filtering, and sorting. By the end of this tutorial, you'll be able to build an API with the ability to fetch specific subsets of data based on parameters like category, brand, and price, as well as control how much data is returned and how it’s ordered.

Setting Up the Project

To get started, make sure you have Node.js installed. If you don’t, download and install it from nodejs.org. Then, follow the steps below to set up the project.

Step 1: Initialize the Project

First, create a new directory for your project and initialize it as a Node.js project:

mkdir product-api
cd product-api
npm init -y

This will create a package.json file, which will keep track of your project dependencies.

Step 2: Install Express and Mongoose

Next, you’ll need to install Express (a fast web framework for Node.js) and Mongoose (to interact with MongoDB). We’ll also install nodemon to automatically restart the server during development.

npm install express mongoose
npm install --save-dev nodemon

Now, update your package.json to use nodemon for easier development by modifying the scripts section:

"scripts": {
  "start": "nodemon index.js"
}

Step 3: Set Up MongoDB

For this tutorial, we’ll assume that you have MongoDB installed. You can run a local instance or use MongoDB Atlas for a cloud-based solution.

Building the API

Now that the project is set up, we can start building the API.

Step 1: Set Up the Express Server

Create a file named index.js and set up a basic Express server:

const express = require('express');
const mongoose = require('mongoose');

const app = express();

// Middleware for parsing JSON
app.use(express.json());

// Connect to MongoDB
mongoose.connect('mongodb://localhost:27017/products', {
  useNewUrlParser: true,
  useUnifiedTopology: true,
}).then(() => {
  console.log('Connected to MongoDB');
}).catch((error) => {
  console.error('Error connecting to MongoDB:', error);
});

// Start the server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});

Step 2: Define the Product Schema

Next, define a Mongoose schema for the products. In this example, each product will have a name, category, brand, price, and stock quantity.

Create a new file called models/Product.js:

const mongoose = require('mongoose');

const productSchema = new mongoose.Schema({
  name: String,
  category: String,
  brand: String,
  price: Number,
  stock: Number
});

module.exports = mongoose.model('Product', productSchema);

Step 3: Create the API Routes

We’ll create a route to retrieve products with pagination, filtering, and sorting options.

In index.js, set up the /products endpoint:

const Product = require('./models/Product');

// Get products with pagination, filtering, and sorting
app.get('/products', async (req, res) => {
  try {
    const { page = 1, limit = 10, sortBy = 'name', order = 'asc', category, brand, price_min, price_max } = req.query;

    // Build filter object
    const filter = {};
    if (category) filter.category = category;
    if (brand) filter.brand = brand;
    if (price_min || price_max) {
      filter.price = {};
      if (price_min) filter.price.$gte = price_min;
      if (price_max) filter.price.$lte = price_max;
    }

    // Set up sorting
    const sortOrder = order === 'asc' ? 1 : -1;

    // Fetch filtered, sorted, and paginated products
    const products = await Product.find(filter)
      .sort({ [sortBy]: sortOrder })
      .skip((page - 1) * limit)
      .limit(Number(limit));

    // Get total product count for pagination
    const totalProducts = await Product.countDocuments(filter);

    res.json({
      products,
      totalPages: Math.ceil(totalProducts / limit),
      currentPage: Number(page),
      totalProducts
    });
  } catch (error) {
    res.status(500).json({ message: 'Error fetching products', error });
  }
});

Step 4: Testing Pagination, Filtering, and Sorting

With the /products endpoint set up, let’s test it. First, seed your database with some sample data. You can create a script to add a few products to MongoDB:

const Product = require('./models/Product');
const mongoose = require('mongoose');

mongoose.connect('mongodb://localhost:27017/products', {
  useNewUrlParser: true,
  useUnifiedTopology: true,
});

const products = [
  { name: 'iPhone 12', category: 'electronics', brand: 'apple', price: 999, stock: 50 },
  { name: 'Samsung Galaxy S21', category: 'electronics', brand: 'samsung', price: 799, stock: 100 },
  { name: 'Sony WH-1000XM4', category: 'electronics', brand: 'sony', price: 350, stock: 200 },
  { name: 'MacBook Pro', category: 'electronics', brand: 'apple', price: 2499, stock: 30 },
  // Add more sample products here...
];

Product.insertMany(products)
  .then(() => {
    console.log('Products added successfully');
    mongoose.connection.close();
  })
  .catch(error => {
    console.error('Error adding products:', error);
  });

After running this script, your MongoDB database will have sample data to work with.

Now, you can test your API using a tool like Postman or curl. Here are some example requests:

1. Basic Pagination

Get the first page of products, with 10 products per page:

GET /products?page=1&limit=10

2. Filtering by Category and Brand

Fetch products that are in the "electronics" category and manufactured by "apple":

GET /products?category=electronics&brand=apple

3. Price Filtering

Get products that are priced between $500 and $1000:

GET /products?price_min=500&price_max=1000

4. Sorting

Sort the products by price in descending order:

GET /products?sortBy=price&order=desc

5. Combined Query

Fetch "apple" products in the "electronics" category, priced above $500, sorted by price in ascending order:

GET /products?category=electronics&brand=apple&price_min=500&sortBy=price&order=asc

Conclusion: Building a Flexible API with Node.js and Express

In this tutorial, we built a flexible API using Node.js and Express that supports pagination, filtering, and sorting. These features allow your API clients to retrieve data efficiently, ensuring they can access exactly what they need without overloading the server or database.

By implementing pagination, you avoid overwhelming the client with too much data at once. Filtering helps narrow down the results based on specific criteria (like category, brand, or price range), while sorting allows users to order the data based on their preferences.

With this foundation in place, you can easily expand your API by adding more features or optimizing the performance further. Keep these principles in mind as you continue to build more advanced APIs for your projects.