How to manage NextJS/Typescript Static Page Optimization

​

The Problem

​ I recently ran into an issue regarding slow build times and decreased page performance for static page generation on a NextJS/Typescript project. Lets first start with some context. Nearly every page in the project utilizes getStaticPaths and getStaticProps, all involved in fetching necessary page data at build time (i.e. next build) to generate the static webpages. During the build process we had a tryReadOrBuild style function called from getStaticPaths and getStaticProps that would either read from a json cache file or if the file did not exist it would fetch the data and create a new json cache file. When the cache file was created it was fetching data from both Shopify and Contentful and merging the data into one large hashtable.

In the end the file ended up being around ~200MB, which means each page size would require at least that much memory. And that was just test data, imagine the catalog will eventually grow. Not only did build times slow down drastically but once the page was built, since the file size was so large the performance of the page itself actually declined. This ended up being the complete opposite result of what you would want to benefit from for static generation. Build times being 15+ minutes which slowed development and page performance halting to unnacceptable load times.

The Solution

​ Since the large cache file contents had each dataset keyed, a simple solution would be to break these files up into smaller files using the key as the file name and the value associated to that key as the contents of the newer and smaller cache file. Multiple smaller files versus one large cache file made fs.readFile and fs.writeFile significantly quicker than the former process. Our largest cache file now only weighs in at 87.8MB with most other cache files only containing about ~75KB of data. What a difference in size! Now each page fetches the smaller cache file(s) vs each page demanding ~200MB which more times than not included data the page did not even require.

The other part of the problem was that getStaticProps and getStaticPaths was calling a function that would either attempt to read or build the cache file on the fly during the next build process. Another potential reason why next build was potentially sluggish. So to further improve on this lets build these smaller cache files prior to even running next build. That way when next build now runs it only has read the pre-built cache files. ​

The Implementation

​ Now that we have our solution planned out lets get down to the nitty gritty. First step is getting our new build cache file function to run prior to next build in a NextJS/Typescript environment. This first step is actually a little tricky mostly because running node scripts in a NextJS/Typescript project is not as simple as it first appears. Lets first look at our old package.json file script commands: ​

"scripts": {
    "dev": "rm -rf ./lib/useCache/index.json && next dev",
    "build": rm -rf ./lib/useCache/index.json && next build",
    "start": "next start -p $PORT",
    "test": "jest",
    "postinstall": "touch ./lib/useCache/index.json"
},

​ now becomes: ​

"scripts": {
    "dev": "npm run prebuild && next dev",
    "build": "npm run prebuild && next build",
    "prebuild": "rm -rf ./lib/useCache/*.json && tsc --project lib-tsconfig.json && node -r dotenv/config ./scripts/prebuild dotenv_config_path=.env.local",
    "start": "next start -p $PORT",
    "test": "jest",
    "postinstall": "touch ./lib/useCache/*.json"
},

​ As you can see above "prebuild" custom script command removes any old cache files, runs the necessary build cache files part of the project through the Typescript compiler so that Node.js can understand and execute it. Our major focus here will be the Typescript compilation in a NextJS/Typescript environment. We needed a separate tsconfig.json file different than NextJS's auto-generated tsconfig.json file. Without it, Node.js couldn't run the prebuild scripts based on NextJS's typescript configuration. ​ Here's NextJS's tsconfig.json: ​

{
    "compilerOptions": {
        "target": "es5",
        "lib": [
            "dom",
            "dom.iterable",
            "esnext"
        ],
        "allowJs": true,
        "skipLibCheck": true,
        "strict": false,
        "forceConsistentCasingInFileNames": true,
        "noEmit": true,p
        "esModuleInterop": true,
        "module": "commonjs",
        "moduleResolution": "node",
        "resolveJsonModule": true,
        "isolatedModules": true,
        "jsx": "preserve",
        "typeRoots": [
            "./types",
            "./node_modules/@types"
        ],
    },
    "include": [
        "next-env.d.ts",
        "**/*.ts",
        "**/*.tsx",
        "jest.env.setup.js" 
    ],
    "exclude": [
        "node_modules"
    ]
}

​ and what you need to compile the prebuild scripts (lib-tsconfig.json): ​

{
    "compilerOptions": {
        "target": "es5",
        "outDir": "./prebuild",
        "esModuleInterop": true,
        "strict": false,
        "moduleResolution": "node",
        "module": "commonjs",
        "skipLibCheck": true,
        "typeRoots": ["./types", "./node_modules/@types"]
    },
    "include": ["modules", "lib"],
    "exclude": ["pages", "hooks", "components", "**/*test.ts", "**/*spec.ts"]
}

​ As you can see we targeted the above file with tsc --project lib-tsconfig.json ​ So our prebuild script now executes the TS compiled output in ./prebuild. To paint a picture of that, here's our prebuild script (for our case located at ./scripts/prebuild): ​

// Import TS compiled prebuild here via CommonJS
const { buildCacheFiles } = require('../prebuild/lib/useCache/index.js');
​
(async function main() {
    try {
        await buildCacheFiles();
    } catch (error) {
        console.error('Error:', error.message);
    } finally {
        return process.exit();
    }
})();

​ When we go to run the prebuild scripts as you see, we also have to load any necessary environment variables with the dotenv package: ​ node -r dotenv/config ./scripts/prebuild dotenv_config_path=.env.local" ​ You'll also probably want to .gitignore these prebuild scripts and the subsequent cache files: ​

/prebuild
​
# build cache
/lib/useCache/*.json

​

The Result

​ In the end, we were able to knock down our build time from ~15+ minutes to ~5 minutes. What a difference! As this is statically generating 100s of pages at build time.