Integrating Pagefind Full-Text Search Engine into a Next.js SSG Blog

What is Pagefind?

Pagefind is a recently developed full-text search engine that runs on the web. The official description states:

Pagefind is a fully static search library that aims to perform well on large sites, while using as little of your users' bandwidth as possible, and without hosting any infrastructure.Pagefind runs after Hugo, Eleventy, Jekyll, Next, Astro, SvelteKit, or any other website framework. The installation process is always the same: Pagefind only requires a folder containing the built static files of your website, so in most cases no configuration is needed to get started.

As you can see from Pagefind/pagefind⭐ 5.4kupdated: 2026-07-30, it is written in (WebAssembly). The stemming -> indexing -> search pipeline appears to be WebAssembly-based.

Japanese Language Support

Multilingual support is available. Japanese support is limited, but apparently Chinese stemming is applied instead of whitespace-based stemming. The supported language is determined by referencing the HTML lang attribute.

Also, segmentation during search is not supported, so tokenization of compound words doesn't happen at search time. You need to do whitespace splitting yourself. Stemming details are written here, but I'm not familiar enough to fully understand the details. It doesn't seem to use dictionaries and appears to be rule-based.

Currently when indexing, Pagefind does not support stemming for specialized languages, but does support segmentation for words not separated by whitespace.s Pagefind does not yet support segmentation of the search query, so searching in the browser requires that words in the search query are separated by whitespace. In practice, this means that on a page tagged as a zh- language, 每個月都 will be indexed as the words 每個, 月, and 都. When searching in the browser, searching for 每個, 月, or 都 individually will work. Additionally, searching 每個 月 都 will return results containing each word in any order, and searching "每個 月 都" in quotes will match 每個月都 exactly. Searching for 每個月都 will return zero results, as Pagefind is not able to segment it into words in the browser. Work to improve this is underway and will hopefully remove this limitation in the future.

Search Algorithm

I couldn't find information about the search algorithm on the official page, but reading the code, it appears to use BM25 as of (2024/09/23).

CloudCannon/pagefind/pagefind_web/src/search.rs
fn calculate_bm25_word_score(
    BM25Params {
        weighted_term_frequency,
        document_length,
        average_page_length,
        total_pages,
        pages_containing_term,
        length_bonus,
    }: BM25Params,
    ranking: &RankingWeights,
) -> f32 {

Implementation in Next.js

This assumes SSG. I'll skip the Next.js configuration details, but output: "export" is required.

Setup

pnpm -i -D pagefind npm-run-all

npm-run-all is used as a task runner.

{
  "scripts": {
    "build": "run-s build:next build:pagefind",
    "build:next": "next build",
    "build:pagefind": "pagefind --site out",
    "dev": "next dev",
    "dev-pagefind": "pagefind --site out --output-path ./public/pagefind"
  }
}

In the pagefind CLI, specify the out directory generated by next build under --site. This generates the index, pagefind.js, etc. under out/pagefind/ (unless --output-path is specified).

For reference, roughly the following is output:

Details
out/pagefind/
├── fragment/
│  ├── ja_1e5d60d.pf_fragment
|   ...
├── index/
│  ├── ja_4dc97e6.pf_index
|   ...
├── pagefind-entry.json
├── pagefind-highlight.js
├── pagefind-modular-ui.css
├── pagefind-modular-ui.js
├── pagefind-ui.css
├── pagefind-ui.js
├── pagefind.ja_5f3319f7c9.pf_meta
├── pagefind.ja_e8a5abf83a.pf_meta
├── pagefind.js
└── wasm.unknown.pagefind

Also, pagefind.js needs to be loadable in the dev environment as well, so during development, we generate it under public/pagefind. Since we don't want this in git, add it to .gitignore.

.gitignore

# pagefind
public/pagefind

Writing the Component

  • Use useEffect to dynamically import /pagefind/pagefind.js.
    • This path should match the output path specified in the pagefind CLI.
  • Zod is used for type-safe result retrieval.
import React, { useEffect, useState } from "react";

import { z } from "zod";

const pagefindResultSchema = z.object({
	url: z.string().transform((url) => url.replace(".html", "")),
	excerpt: z.string(),
	meta: z.object({
		title: z.string().optional(),
		image: z.string().optional(),
	}),
});

type PagefindResult = z.infer<typeof pagefindResultSchema>;

declare global {
	interface Window {
		// eslint-disable-next-line @typescript-eslint/no-explicit-any
		pagefind: any;
	}
}

const pagefindResultSchema = z.object({
	// Remove the trailing .html added during export.
	url: z.string().transform((url) => url.replace(".html", "")),
	excerpt: z.string(),
	meta: z.object({
		title: z.string().optional(),
		image: z.string().optional(),
	}),
});

type PagefindResult = z.infer<typeof pagefindResultSchema>;

declare global {
	interface Window {
		// eslint-disable-next-line @typescript-eslint/no-explicit-any
		pagefind: any;
	}
}

export default function Search() {
	const [query, setQuery] = useState("");
	const [results, setResults] = useState<PagefindResult[]>([]);

	useEffect(() => {
		async function loadPagefind() {
			if (typeof window.pagefind === "undefined") {
				try {
					// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
					window.pagefind = await import(
						// @ts-expect-error @types of pagefind are not available
						// eslint-disable-next-line import/no-unresolved
						/* webpackIgnore: true */ "/pagefind/pagefind.js"
					);
				} catch (e) {
					console.error(e);
					window.pagefind = { search: () => ({ results: [] }) };
				}
			}
		}
		// eslint-disable-next-line @typescript-eslint/no-floating-promises
		loadPagefind();
	}, []);

	async function handleSearch() {
		if (!window.pagefind) {
			return;
		}

		// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-unsafe-call
		const search = await window.pagefind.search(query);

		// eslint-disable-next-line @typescript-eslint/no-unsafe-call, @typescript-eslint/no-unsafe-member-access, @typescript-eslint/no-explicit-any, @typescript-eslint/no-unsafe-return
		const results = await Promise.all(search.results.map((r: any) => r.data()));

		setResults(z.array(pagefindResultSchema).parse(results));
	}

	return (
		<div>
			<form
				onSubmit={(e) => {
					e.preventDefault();
					// eslint-disable-next-line @typescript-eslint/no-misused-promises, @typescript-eslint/no-floating-promises
					handleSearch();
				}}
			>
				<input
					type="text"
					value={query}
					placeholder="Search articles..."
					onChange={(e) => setQuery(e.target.value)}
				/>
				<button type="submit">Search</button>
			</form>

			{results.map((result) => (
				<div key={result.url}>
					<h2>
						<Link href={result.url}>{result.meta.title ?? "Untitled"}</Link>
					</h2>
					<div dangerouslySetInnerHTML={{ __html: result.excerpt }} />
				</div>
			))}
		</div>
	);
}

For type information, refer to the following. I only used the parts that seemed necessary.

CloudCannon/pagefind/pagefind_web_js/types/index.d.ts
    type PagefindSearchFragment = {
        /** Pagefind's processed URL for this page. Will include the baseUrl if configured */
        url: string,
        /** Pagefind's unprocessed URL for this page */
        raw_url?: string
        /** The full processed content text of this page */
        content: string,
        /** Internal type — ignore for now */
        raw_content?: string;
        /** The processed excerpt for this result, with matching terms wrapping in `<mark>` elements */
        excerpt: string,
        /**
         * What regions of the page matched this search query?
         * 
         * Precalculates based on h1->6 tags with IDs, using the text between each.
         */
        sub_results: PagefindSubResult[],
        /** How many total words are there on this page? */
        word_count: number,
        /** The locations of all matching words in this page */
        locations: number[],
        /** 
         * The locations of all matching words in this page,
         * paired with data about their weight and relevance to this query
         */
        weighted_locations: PagefindWordLocation[],
        /** The filter keys and values this page was tagged with */
        filters: Record<string, string[]>
        /** The metadata keys and values this page was tagged with */
        meta: Record<string, string>,
        /**
         * The raw anchor data that Pagefind used to generate sub_results.
         * 
         * Contains _all_ elements that had IDs on the page, so can be used to
         * implement your own sub result calculations with different semantics.
         */
        anchors: PagefindSearchAnchor[],
    }

Specifying Where to Index

You can control where indexing occurs by specifying data-pagefind-* tags.

The most straightforward example is specifying data-pagefind-body, which causes only the content within main to be indexed:

<body>
    <main data-pagefind-body>
        <h1>Condimentum Nullam</h1>
        <p>Nullam id dolor id nibh ultricies.</p>
    </main>
    <aside>
        This content will not be indexed.
    </aside>
</body>

You can also exclude specific sections from indexing by specifying data-pagefind-ignore.

For more details, see:

Conclusion

I was impressed by how easy it was to add a search engine to an SSG site. This makes it seem like you can host a blog on Cloudflare or similar services with all the necessary features.

Create an issue on GitHub about this article

Read Next