Blog

Adding Pagination and Search to our Gatsby Blog

Taylor Bantle
WEB
7 min read

DoltHub is a place on the internet to share, discover, and collaborate on Dolt databases. Our blog is a separate Gatsby application. Each page is created from a markdown file, which includes metadata that can be queried through the GraphQL API from our React components.

We publish three blog posts a week and our blog index page was getting long. We recently added pagination and search to our blog to make it easier to navigate. We also have some useful information and links in our footer that are difficult to reach without pagination.

There are a lot of great resources out there in both the Gatsby docs and other blogs for adding pagination and search to a Gatsby site. However, there were some nuances and roadblocks I encountered along the way. A blog that explained what I learned could be useful for others, so here we are.

Adding pagination

I started with adding pagination, using Gatsby's docs as a reference.

First we had to modify our blog list index GraphQL query to include the skip and limit parameters.

// pages/index.tsx

export const query = graphql`
  query BlogList($limit: Int!, $skip: Int!) {
    allMarkdownRemark(
      sort: { fields: frontmatter___date, order: DESC }
      limit: $limit
      skip: $skip
    ) {
      nodes {
        ...NodeForBlogList
      }
    }
    allSitePage {
      nodes {
        path
      }
    }
  }
`;

Next, we needed to modify our gatsby-node.js file to create the paginated pages. Because we just had one page for our blog list, the component lived in our pages/index.tsx file. In order to use this component for every paginated blog list page, we needed to first move this component to a reusable templates/BlogList.tsx file.

Our gatsby-node.js has two main methods, onCreateNode and createPages. We need to add some logic to the end of createPages to create a page for each paginated blog list page. The blog index is page one, and every following page with have the path /page/2, /page/3, etc.

// gatsby-node.js

exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions;

  const posts = result.data.allMarkdownRemark.edges;
  // [Create each blog post page]

  // Create paginated blog list pages
  const postsPerPage = 20;
  const numPages = Math.ceil(posts.length / postsPerPage);
  Array.from({ length: numPages }).forEach((_, i) => {
    const firstPage = i === 0;
    const currentPage = i + 1;
    createPage({
      path: firstPage ? "/" : `/page/${currentPage}`,
      component: path.resolve("./src/templates/BlogList.tsx"),
      context: {
        limit: postsPerPage,
        skip: i * postsPerPage,
        numPages,
        currentPage,
      },
    });
  });
};

Now that paths for our paginated pages exist, we need to add buttons to navigate between pages. Our blog is important for search term rankings for Dolt-relevant keywords, so first I did some research into how pagination affects SEO. This article has some great information about it.

We decided to go with showing five page numbers at a time with next and previous links, like this:

Page buttons

We used the pageContext prop to pass down the relevant information to our page buttons component, which ended up looking like this:

// components/PageButtons.tsx

import { Link } from "gatsby";
import React from "react";
import { SitePageContext } from "../../graphql-schema";

type Props = {
  pageContext: SitePageContext;
};

export default function PageButtons({ pageContext }: Props) {
  const { numPages, currentPage } = pageContext;
  if (!numPages || !currentPage) return null;

  const isFirst = currentPage === 1;
  const isLast = currentPage === numPages;
  const prev = currentPage === 2 ? "/" : `/page/${currentPage - 1}`;
  const next = currentPage + 1;

  const start = getStart(currentPage, numPages);
  const nums = Array.from({ length: 5 }).map((_, i) => i + start);

  return (
    <div>
      <div>
        {!isFirst && (
          <span>
            <Link to={prev} rel="prev">
              Previous
            </Link>
          </span>
        )}
        <span>
          <ul>
            {nums.map((num) => (
              <li key={num} className={num === currentPage ? "num-active" : ""}>
                <Link to={num === 1 ? "/" : `/page/${num}`}>{num}</Link>
              </li>
            ))}
          </ul>
        </span>
        {!isLast && (
          <span>
            <Link to={`/page/${next}`} rel="next">
              Next
            </Link>
          </span>
        )}
      </div>
    </div>
  );
}

Now that our blog has pagination, we can't use command+F to search for older blogs. We needed to add our own search before we could release pagination.

Adding search

Adding search to our blog was a little more complicated. Gatsby has a doc about adding search, but it was not immediately clear which path was best. I ended up trying out a few things before finding one that worked.

First, I tried the Gatsby plugin gatsby-plugin-local-search. You can use this plugin with either FlexSearch or Lunr.

FlexSearch

I first tried FlexSearch because Gatsby recommended it and it was used in the example in their docs. However, after I successfully set up the config and queries and installed react-use-flexsearch I started to run into some problems. react-use-flexsearch has not been updated since March 2020. Not only does it not play well with Typescript, but the most recent version of FlexSearch did not work at all with the provided hook. I looked into alternatives and found a typed version, but this also didn't work with the newest version of Typescript, and I couldn't find the source code in order to adopt my own version to modify. I spent some time writing my own hook, but it became too time consuming so I decided to try alternatives.

Lunr

It was pretty easy to switch to Lunr from FlexSearch. I switched the engine field in my gatsby config for the gatsby-plugin-local-search plugin to "lunr" and installed the recommended hook, react-lunr. This played well with Typescript and I was able to start searching right away. However once I showed the team, we realized their search matching wasn't giving the results we expected. After some digging in the Lunr docs, I realized Lunr only does word matching, not phrase matching. So searching for "version controlled database" gave us results for "version" OR "controlled" OR "database" and not the phrase "version controlled database". It didn't seem like there was a way to fix this, so I was back to the beginning.

I eventually found success using js-search. This took a second for me to make work for our setup, because the example in the docs does not use the Gatsby GraphQL API like we do. I was happy to see there's a corresponding @types/js-search package, so it works well with Typescript.

First, I installed js-search and created a custom hook that handles searching:

// util/useJsSearch.ts

import * as JsSearch from "js-search";
import {
  Maybe,
  NodeForBlogIndexFragment,
  SitePageContextAllBlogs,
} from "../../graphql-schema";

type ReturnType = {
  search: (q: string) => NodeForBlogIndexFragment[];
};

export default function useJsSearch(
  blogs?: Maybe<Array<Maybe<SitePageContextAllBlogs>>>
): ReturnType {
  // Search configuration
  const dataToSearch = new JsSearch.Search("id");
  dataToSearch.indexStrategy = new JsSearch.PrefixIndexStrategy();
  dataToSearch.sanitizer = new JsSearch.LowerCaseSanitizer();
  dataToSearch.searchIndex = new JsSearch.TfIdfSearchIndex("id");

  // Fields to search
  dataToSearch.addIndex(["frontmatter", "title"]);
  dataToSearch.addIndex(["frontmatter", "author"]);
  dataToSearch.addIndex("excerpt");

  // Map types and filter out empty nodes
  const mapNodes = mapBlogsToIndexNodes(blogs);
  dataToSearch.addDocuments(mapNodes);

  const search = (query: string) =>
    dataToSearch.search(query) as NodeForBlogIndexFragment[];

  return { search };
}

Then I used the hook in our BlogList template:

// templates/BlogList.tsx

import { PageProps } from "gatsby";
import React, { useEffect, useState } from "react";
import { BlogPaginatedQuery, SitePageContext } from "../../graphql-schema";
import BlogPostExcerpt from "../components/BlogPostExcerpt";
import Layout from "../components/Layout";
import PageButtons from "../components/PageButtons";
import useJsSearch from "../util/useJsSearch";

type Props = PageProps & {
  data: BlogPaginatedQuery;
  pageContext: SitePageContext;
};

export default function BlogList({ data, pageContext, location }: Props) {
  const { search } = useJsSearch(pageContext.allBlogs);

  const params = new URLSearchParams(location.search.slice(1));
  const q = params.get("q") ?? "";
  const results = search(q);
  const posts = q ? results : data.allMarkdownRemark.nodes;
  const nodes = posts.filter((n) => new Date(n.frontmatter?.date) < new Date());

  return (
    <Layout>
      <ol>
        {nodes.map((node, i) => (
          <BlogPostExcerpt data={node} />
        ))}
      </ol>
      {!initialQuery && <PageButtons pageContext={pageContext} />}
    </Layout>
  );
}

We ran into some problems with the blog list not updating on clearing the search or navigating directly to a searched term (i.e. /?q=search), so we used useEffect to ensure our search state is always up to date.

const [blogs, setBlogs] = useState(data.allMarkdownRemark.nodes);
const [searched, setSearched] = useState(false);
const [initialQuery, setInitialQuery] = useState("");

// Handles query state and prevents unnecessary rerendering
useEffect(() => {
  const params = new URLSearchParams(location.search.slice(1));
  const q = params.get("q") ?? "";
  // Check if we have searched
  if (q !== initialQuery) {
    setSearched(false);
  }
  setInitialQuery(q);
  // If no query, reset blogs
  if (!q) {
    setBlogs(data.allMarkdownRemark.nodes);
    return;
  }
  // If query exists and we haven't searched yet, execute search
  if (q && !searched) {
    const results = search(q);
    setBlogs(results);
    setSearched(true);
  }
}, [
  searched,
  data.allMarkdownRemark.nodes,
  search,
  location.search,
  initialQuery,
]);

We needed a way to execute a search, so I created a new Search component and added it to our BlogList template:

// components/Search.tsx

import { Link, navigate } from "gatsby";
import React, { SyntheticEvent, useEffect, useState } from "react";

type Props = {
  initialQuery: string;
  numResults: number;
};

export default function Search(props: Props) {
  const [query, setQuery] = useState(props.initialQuery);

  useEffect(() => {
    setQuery(props.initialQuery);
  }, [props.initialQuery]);

  const onSubmit = async (e: SyntheticEvent) => {
    e.preventDefault();
    try {
      await navigate(`/?q=${query}`);
    } catch (err) {
      console.error(err);
    }
  };

  return (
    <div>
      <form onSubmit={onSubmit}>
        <input
          type="text"
          placeholder="Search blogs"
          value={query}
          onChange={(e) => setQuery(e.target.value)}
        />
        {!!props.initialQuery && <Link to="/">&#10005;</Link>}
        {props.initialQuery && (
          <div>
            Found {props.numResults} matching article
            {props.numResults === 1 ? "" : "s"}.
          </div>
        )}
      </form>
    </div>
  );
}

Our search results for phrases are more accurate now and our blog is easier to navigate.

Search result

Conclusion

As our blog continues to grow, pagination and search became more important to improve the user experience. We have further work to do to paginate search results, so stay tuned for a follow up blog. You can subscribe to our mailing list or join us on Discord to get updates or chat with us about Dolt or any of the above.

SHARE

Blog
Dolt is open source
JOIN THE DATA EVOLUTION

Get started with Dolt

Or join our mailing list to get product updates.