How we converted our Node.js library to Deno (using Deno)

Node.js vs Deno​

There are a few key differences between the Node.js and Deno runtimes that we will have to take into account when adapting our library written for Node.js:

1. TypeScript support: Deno can directly execute TypeScript files, while Node.js can only run JavaScript code.

2. Module resolution: By default, Node.js imports modules in the CommonJS format and expects require/module.exports syntax. It also has a complex resolution algorithm, which will load plain module names like "react" from node_modules, append .js or .json as needed to extensionless imports, and import the index.js file if the import path is a directory.

   Deno simplifies this dramatically. It uses the ECMAScript module syntax import and export, also known as “ES modules” or just “ESM”. This syntax is also used by TypeScript. All imports must be either a) a relative path with an explicit file extensions or b) a URL.

   This means there is no node_modules or package manager like npm or yarn; external modules are simply imported directly via a URL from a publicly available code repository, like deno.land/x or GitHub.

3. Standard library: Node.js has a builtin set of standard modules like fs, path, crypto, and http, among others. These modules can be directly required with require('fs') and their names are reserved by Node.js. By contrast, the Deno standard library is imported via URL from the https://deno.land/std/ registry. The functionality of the two standard libraries also differs, with Deno abandoning some old or outdated Node.js APIs, introducing a new standard library (inspired by that of Go), and uniformly supporting modern JavaScript features like Promises (whereas many Node.js APIs still rely on the older callback style).

4. Built-in globals: Deno contains all of its core APIs under a single global variable called Deno and otherwise only exposes standard web APIs. Unlike Node.js, there is no global Buffer or process variable available.

So how can we work around these differences and get our Node.js library running in Deno as painlessly as possible? Let’s go through these changes one by one.

TypeScript and module syntax​

Fortunately, we don’t need to worry much about converting CommonJS require/module.exports syntax to ESM import/export. We wrote edgedb-js entirely in TypeScript, which already uses ESM syntax. During compilation, tsc converts our files to plain JavaScript files utilizing CommonJS require syntax. The compiled files are directly consumable by Node.js.

The rest of this post will discuss how we modify our TypeScript source files to a format that is directly consumable by Deno.

Dependencies​

Fortunately edgedb-js doesn’t have any third-party dependencies, so we don’t have to worry about the Deno compatibility of any external libraries. However, we need to replace all imports from the Node.js standard library (e.g. path, fs, etc) with a Deno equivalent.

This task is made far simpler by the existence of the Node.js compatibility module provided by the Deno standard library. This module provides a wrapper over Deno’s standard library that attempts to adhere as faithfully as possible to Node’s API.

import * as crypto from "crypto";
import * as crypto from "https://deno.land/std@0.114.0/node/crypto.ts";

To simplify things, we moved all imports of Node.js APIs to a single file called adapter.node.ts and re-export only the functionality we need.

// adapter.node.ts
import * as path from "path";
import * as util from "util";
import * as crypto from "crypto";

export {path, net, crypto};

We then implement the same adapter for Deno in a file called adapter.deno.ts.

// adapter.deno.ts
import * as crypto from "https://deno.land/std@0.114.0/node/crypto.ts";
import path from "https://deno.land/std@0.114.0/node/path.ts";
import util from "https://deno.land/std@0.114.0/node/util.ts";

export {path, util, crypto};

Whenever we need Node.js specific functionality, we import it from adapter.node.ts directly. This way, we can make edgedb-js Deno-compatible by simply rewriting all imports of adapter.node.ts to adapter.deno.ts. As long as these files re-export the same functionality, everything should work as expected.

Practically speaking, how do we actually rewrite these imports? Well, we need to write a simple codemod script. And just to make it a bit more poetic, we’ll write this script using Deno itself.

Writing a Deno-ifier​

Before we get started building, let’s outline what steps this tool needs to do:

• Rewrite the Node.js-style imports to the more explicit style Deno. This includes adding the .ts file extension and adding /index.ts to all directory imports.

• Swapping out all imports from adapter.node to adapter.deno.ts.

• Inject Node.js globals like process and Buffer into the Deno-ified code. While we could simply export these variables from our adapters, we’d have to refactor our Node.js files to explicitly import them. To simplify things, we’ll detect where Node.js globals are used and inject an import as needed.

• Rename the src directory to _src to denote that it holds the internals of edgedb-js and shouldn’t be imported directly

• Move the main src/index.node.ts file to the project root and rename it mod.ts. This is idiomatic in Deno. (Note: the naming of index.node.ts here doesn’t indicate that it’s Node-specific; it’s indented to differentiate it from index.browser.ts, which exports all browser-compatible parts of edgedb-js.)

import {walk} from "https://deno.land/std@0.114.0/fs/mod.ts";

const sourceDir = "./src";
for await (const entry of walk(sourceDir, {includeDirs: false})) {
  // iterate through all files
}

const sourceDir = "./src";
const destDir = "./edgedb-deno";
const pathRewriteRules = [
  {match: /^src\/index.node.ts$/, replace: "mod.ts"},
  {match: /^src\//, replace: "_src/"},
];

const sourceFilePathMap = new Map<string, string>();

for await (const entry of walk(sourceDir, {includeDirs: false})) {
  const sourcePath = entry.path;
  sourceFilePathMap.set(sourcePath, resolveDestPath(sourcePath));
}

function resolveDestPath(sourcePath: string) {
  let destPath = sourcePath;
  // apply all rewrite rules
  for (const rule of pathRewriteRules) {
    destPath = destPath.replace(rule.match, rule.replace);
  }
  return join(destDir, destPath);
}

import {createRequire} from "https://deno.land/std@0.114.0/node/module.ts";

const require = createRequire(import.meta.url);
const ts = require("typescript");

import {walk, ensureDir} from "https://deno.land/std@0.114.0/fs/mod.ts";
import {createRequire} from "https://deno.land/std@0.114.0/node/module.ts";

const require = createRequire(import.meta.url);
const ts = require("typescript");

for (const [sourcePath, destPath] of sourceFilePathMap) {
  compileFileForDeno(sourcePath, destPath);
}

async function compileFileForDeno(sourcePath: string, destPath: string) {
  const file = await Deno.readTextFile(sourcePath);
  await ensureDir(dirname(destPath));

  // if file ends with '.deno.ts', copy the file unchanged
  if (destPath.endsWith(".deno.ts")) return Deno.writeTextFile(destPath, file);
  // if file ends with '.node.ts', skip file
  if (destPath.endsWith(".node.ts")) return;

  // parse the source file using the typescript Compiler API
  const parsedSource = ts.createSourceFile(
    basename(sourcePath),
    file,
    ts.ScriptTarget.Latest,
    false,
    ts.ScriptKind.TS
  );
}

const parsedSource = ts.createSourceFile(/*...*/);

const rewrittenFile: string[] = [];
let cursor = 0;
parsedSource.forEachChild((node: any) => {
  if (
    (node.kind === ts.SyntaxKind.ImportDeclaration ||
      node.kind === ts.SyntaxKind.ExportDeclaration) &&
    node.moduleSpecifier
  ) {
    const pos = node.moduleSpecifier.pos + 2;
    const end = node.moduleSpecifier.end - 1;
    const importPath = file.slice(pos, end);

    rewrittenFile.push(file.slice(cursor, pos));
    cursor = end;

    // replace the adapter import with Deno version
    let resolvedImportPath = resolveImportPath(importPath, sourcePath);
    if (resolvedImportPath.endsWith("/adapter.node.ts")) {
      resolvedImportPath = resolvedImportPath.replace(
        "/adapter.node.ts",
        "/adapter.deno.ts"
      );
    }

    rewrittenFile.push(resolvedImportPath);
  }
});

rewrittenFile.push(file.slice(cursor));

export {Buffer} from "https://deno.land/std@0.114.0/node/buffer.ts";
export {process} from "https://deno.land/std@0.114.0/node/process.ts";

const sourceDir = "./src";
const destDir = "./edgedb-deno";
const pathRewriteRules = [
  {match: /^src\/index.node.ts$/, replace: "mod.ts"},
  {match: /^src\//, replace: "_src/"},
];
const injectImports = {
  imports: ["Buffer", "process"],
  from: "src/globals.deno.ts",
};

// ...

const rewrittenFile: string[] = [];
let cursor = 0;
let isFirstNode = true;
parsedSource.forEachChild((node: any) => {
  if (isFirstNode) {  // only run once per file
    isFirstNode = false;

    const neededImports = injectImports.imports.filter((importName) =>
      parsedSource.identifiers?.has(importName)
    );

    if (neededImports.length) {
      const imports = neededImports.join(", ");
      const importPath = resolveImportPath(
        relative(dirname(sourcePath), injectImports.from),
        sourcePath
      );
      const importDecl = `import {${imports}} from "${importPath}";\n\n`;

      const injectPos = node.getLeadingTriviaWidth?.(parsedSource) ?? node.pos;
      rewrittenFile.push(file.slice(cursor, injectPos));
      rewrittenFile.push(importDecl);
      cursor = injectPos;
    }
  }

try {
  await Deno.remove(destDir, {recursive: true});
} catch {}

const sourceFilePathMap = new Map<string, string>();
for (const [sourcePath, destPath] of sourceFilePathMap) {
  // rewrite file
  await Deno.writeTextFile(destPath, rewrittenFile.join(""));
}

# .github/workflows/deno-release.yml
name: Deno Release
on:
  push:
    branches:
      - master
jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout edgedb-js
        uses: actions/checkout@v2
      - name: Checkout edgedb-deno
        uses: actions/checkout@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          repository: edgedb/edgedb-deno
          path: edgedb-deno
      - uses: actions/setup-node@v2
      - uses: denoland/setup-deno@v1
      - name: Install deps
        run: yarn install
      - name: Get version from package.json
        id: package-version
        uses: martinbeentjes/npm-get-version-action@v1.1.0
      - name: Write version to file
        run: echo "${{ steps.package-version.outputs.current-version}}" > edgedb-deno/version.txt
      - name: Compile for Deno
        run: deno run --unstable --allow-env --allow-read --allow-write tools/compileForDeno.ts
      - name: Push to edgedb-deno
        run: cd edgedb-deno && git add . -f && git commit -m "Build from $GITHUB_SHA" && git push

Wrapping up​

This is a broadly generalizable pattern for converting an existing Node.js module to Deno. Refer to the edgedb-js repo for the full Deno compilation script, cross- workflow.