Merge pull request #4270 from microsoft/octogonz/rush-sdk-proxy

[rush-sdk] Introduce a proxy API for customizing how rush-lib is loaded

Author: iclanton

build-tests/install-test-workspace/workspace/common/pnpm-lock.yaml

@@ -0,0 +1,10 @@
+{
+  "changes": [
+    {
+      "packageName": "@microsoft/rush",
+      "comment": "@rushstack/rush-sdk now exposes a secondary API for manually loading the Rush engine and monitoring installation progress",
+      "type": "none"
+    }
+  ],
+  "packageName": "@microsoft/rush"
+}

common/changes/@microsoft/rush/octogonz-rush-sdk-proxy_2023-08-04-23-41.json

@@ -0,0 +1,39 @@
+## API Report File for "@rushstack/rush-sdk"
+
+> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/).
+
+```ts
+
+/// <reference types="node" />
+
+// @public
+export interface ILoadSdkAsyncOptions {
+    abortSignal?: AbortSignal;
+    onNotifyEvent?: SdkNotifyEventCallback;
+    rushJsonSearchFolder?: string;
+}
+
+// @public
+export interface IProgressBarCallbackLogMessage {
+    kind: 'info' | 'debug';
+    text: string;
+}
+
+// @public
+export interface ISdkCallbackEvent {
+    logMessage: IProgressBarCallbackLogMessage | undefined;
+    progressPercent: number | undefined;
+}
+
+// @public
+export class RushSdkLoader {
+    static get isLoaded(): boolean;
+    static loadAsync(options?: ILoadSdkAsyncOptions): Promise<void>;
+}
+
+// @public
+export type SdkNotifyEventCallback = (sdkEvent: ISdkCallbackEvent) => void;
+
+// (No @packageDocumentation comment for this package)
+
+```

common/config/rush/pnpm-lock.yaml

@@ -4,18 +4,110 @@ This is a companion package for the Rush tool. See the [@microsoft/rush](https:/
 
 ⚠ **_THIS PACKAGE IS EXPERIMENTAL_** ⚠
 
-The **@rushstack/rush-sdk** package acts as a lightweight proxy for accessing the APIs of the **@microsoft/rush-lib** engine. It is intended to support three different use cases:
+The **@rushstack/rush-sdk** package acts as a lightweight proxy for accessing the APIs of the **@microsoft/rush-lib** engine. It is intended to support five different use cases:
 
-1. Rush plugins should import from **@rushstack/rush-sdk** instead of **@microsoft/rush-lib**. This gives plugins full access to Rush APIs while avoiding a redundant installation of those packages. At runtime, the APIs will be bound to the correct `rushVersion` from **rush.json**, and guaranteed to be the same **@microsoft/rush-lib** module instance as the plugin host.
+1. **Rush plugins:** Rush plugins should import from **@rushstack/rush-sdk** instead of **@microsoft/rush-lib**. This gives plugins full access to Rush APIs while avoiding a redundant installation of those packages. At runtime, the APIs will be bound to the correct `rushVersion` from **rush.json**, and guaranteed to be the same **@microsoft/rush-lib** module instance as the plugin host.
 
-2. When authoring unit tests for a Rush plugin, developers should add **@microsoft/rush-lib** to their **package.json** `devDependencies`. In this context, **@rushstack/rush-sdk** will resolve to that instance for testing purposes.
+2. **Unit tests:** When authoring unit tests (for a Rush plugin, for example), developers should add **@microsoft/rush-lib** to their **package.json** `devDependencies` and add **@rushstack/rush-sdk** to the regular `dependencies`. In this context, **@rushstack/rush-sdk** will resolve to the locally installed instance for testing purposes.
 
-3. For projects within a monorepo that use **@rushstack/rush-sdk** during their build process, child processes will inherit the installation of Rush that invoked them. This is communicated using the `_RUSH_LIB_PATH` environment variable.
+3. **Rush subprocesses:** For tools within a monorepo that import **@rushstack/rush-sdk** during their build process, child processes will inherit the installation of Rush that invoked them. This is communicated using the `_RUSH_LIB_PATH` environment variable.
 
-4. For scripts and tools that are designed to be used in a Rush monorepo, in the future **@rushstack/rush-sdk** will automatically invoke **install-run-rush.js** and load the local installation. This ensures that tools load a compatible version of the Rush engine for the given branch. Once this is implemented, **@rushstack/rush-sdk** can replace **@microsoft/rush-lib** entirely as the official API interface, with the latter serving as the underlying implementation.
+4. **Monorepo tools:** For scripts and tools that are designed to be used in a Rush monorepo, **@rushstack/rush-sdk** will automatically invoke **install-run-rush.js** and load the local installation. This ensures that tools load a compatible version of the Rush engine for the given branch.
+
+5. **Advanced scenarios:** The secondary `@rushstack/rush-sdk/loader` entry point can be imported by tools that need to explicitly control where **@microsoft/rush-lib** gets loaded from. This API also allows monitoring installation and canceling the operation.  This API is used by the Rush Stack VS Code extension, for example.
 
 The **@rushstack/rush-sdk** API declarations are identical to the corresponding version of **@microsoft/rush-lib**.
 
+## Basic usage
+
+Here's an example of basic usage that works with cases 1-4 above:
+
+```ts
+// CommonJS notation:
+const { RushConfiguration } = require('@rushstack/rush-sdk');
+
+const config = RushConfiguration.loadFromDefaultLocation();
+console.log(config.commonFolder);
+```
+
+```ts
+// TypeScript notation:
+import { RushConfiguration } from '@rushstack/rush-sdk';
+
+const config = RushConfiguration.loadFromDefaultLocation();
+console.log(config.commonFolder);
+```
+
+## Loader API
+
+Here's a basic example of how to manually load **@rushstack/rush-sdk** and monitor installation progress:
+
+```ts
+import { RushSdkLoader, ISdkCallbackEvent } from '@rushstack/rush-sdk/loader';
+
+if (!RushSdkLoader.isLoaded) {
+  await RushSdkLoader.loadAsync({
+    // the search for rush.json starts here:
+    rushJsonSearchFolder: "path/to/my-repo/apps/my-app",
+
+    onNotifyEvent: (event: ISdkCallbackEvent) => {
+      if (event.logMessage) {
+        // Your tool can show progress about the loading:
+        if (event.logMessage.kind === 'info') {
+          console.log(event.logMessage.text);
+        }
+      }
+    }
+  });
+}
+
+// Any subsequent attempts to call require() will return the same instance
+// that was loaded above.
+const rushSdk = require('@rushstack/rush-sdk');
+const config = rushSdk.RushConfiguration.loadFromDefaultLocation();
+```
+
+Here's a more elaborate example illustrating other API features:
+
+```ts
+import { RushSdkLoader, ISdkCallbackEvent } from '@rushstack/rush-sdk/loader';
+
+// Use an AbortController to cancel the operation after a certain time period
+const abortController = new AbortController();
+setTimeout(() => {
+  abortController.abort();
+}, 1000);
+
+if (!RushSdkLoader.isLoaded) {
+  await RushSdkLoader.loadAsync({
+    // the search for rush.json starts here:
+    rushJsonSearchFolder: "path/to/my-repo/apps/my-app",
+
+    abortSignal: abortController.signal,
+
+    onNotifyEvent: (event: ISdkCallbackEvent) => {
+      if (event.logMessage) {
+        // Your tool can show progress about the loading:
+        if (event.logMessage.kind === 'info') {
+          console.log(event.logMessage.text);
+        }
+      }
+
+      if (event.progressPercent !== undefined) {
+        // If installation takes a long time, your tool can display a progress bar
+        displayYourProgressBar(event.progressPercent);
+      }
+    }
+  });
+}
+
+// Any subsequent attempts to call require() will return the same instance
+// that was loaded above.
+const rushSdk = require('@rushstack/rush-sdk');
+const config = rushSdk.RushConfiguration.loadFromDefaultLocation();
+```
+
+
 ## Importing internal APIs
 
 Backwards compatibility is only guaranteed for the APIs marked as `@public` in the official `rush-lib.d.ts` entry point.

common/reviews/api/rush-sdk.api.md

@@ -0,0 +1,20 @@
+{
+  "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json",
+
+  "mainEntryPointFilePath": "<projectFolder>/lib-shim/loader.d.ts",
+
+  "apiReport": {
+    "enabled": true,
+    "reportFolder": "../../../common/reviews/api"
+  },
+
+  "docModel": {
+    "enabled": false,
+    "apiJsonFilePath": "../../../common/temp/api/<unscopedPackageName>.api.json"
+  },
+
+  "dtsRollup": {
+    "enabled": true,
+    "publicTrimmedFilePath": "<projectFolder>/dist/loader.d.ts"
+  }
+}

libraries/rush-sdk/README.md

@@ -10,6 +10,17 @@
   "homepage": "https://rushjs.io",
   "main": "lib-shim/index.js",
   "typings": "dist/rush-lib.d.ts",
+  "exports": {
+    ".": "./lib-shim/index.js",
+    "./loader": "./lib-shim/loader.js"
+  },
+  "typesVersions": {
+    "*": {
+      "loader": [
+        "./dist/loader.d.ts"
+      ]
+    }
+  },
   "scripts": {
     "build": "heft build --clean",
     "_phase:build": "heft run --only build -- --clean",
@@ -29,6 +40,7 @@
     "@rushstack/stream-collator": "workspace:*",
     "@rushstack/ts-command-line": "workspace:*",
     "@rushstack/terminal": "workspace:*",
+    "@types/node": "14.18.36",
     "@types/semver": "7.5.0",
     "@types/webpack-env": "1.18.0"
   }

libraries/rush-sdk/config/api-extractor.json

@@ -0,0 +1,71 @@
+// Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT license.
+// See LICENSE in the project root for license information.
+
+import * as path from 'path';
+import { Import, FileSystem } from '@rushstack/node-core-library';
+import type { EnvironmentVariableNames } from '@microsoft/rush-lib';
+
+export const RUSH_LIB_NAME: '@microsoft/rush-lib' = '@microsoft/rush-lib';
+export const RUSH_LIB_PATH_ENV_VAR_NAME: typeof EnvironmentVariableNames.RUSH_LIB_PATH = '_RUSH_LIB_PATH';
+
+export type RushLibModuleType = Record<string, unknown>;
+
+export interface ISdkContext {
+  rushLibModule: RushLibModuleType | undefined;
+}
+
+export const sdkContext: ISdkContext = {
+  rushLibModule: undefined
+};
+
+/**
+ * Find the rush.json location and return the path, or undefined if a rush.json can't be found.
+ *
+ * @privateRemarks
+ * Keep this in sync with `RushConfiguration.tryFindRushJsonLocation`.
+ */
+export function tryFindRushJsonLocation(startingFolder: string): string | undefined {
+  let currentFolder: string = startingFolder;
+
+  // Look upwards at parent folders until we find a folder containing rush.json
+  for (let i: number = 0; i < 10; ++i) {
+    const rushJsonFilename: string = path.join(currentFolder, 'rush.json');
+
+    if (FileSystem.exists(rushJsonFilename)) {
+      return rushJsonFilename;
+    }
+
+    const parentFolder: string = path.dirname(currentFolder);
+    if (parentFolder === currentFolder) {
+      break;
+    }
+
+    currentFolder = parentFolder;
+  }
+
+  return undefined;
+}
+
+export function _require<TResult>(moduleName: string): TResult {
+  if (typeof __non_webpack_require__ === 'function') {
+    // If this library has been bundled with Webpack, we need to call the real `require` function
+    // that doesn't get turned into a `__webpack_require__` statement.
+    // `__non_webpack_require__` is a Webpack macro that gets turned into a `require` statement
+    // during bundling.
+    return __non_webpack_require__(moduleName);
+  } else {
+    return require(moduleName);
+  }
+}
+
+/**
+ * Require `@microsoft/rush-lib` under the specified folder path.
+ */
+export function requireRushLibUnderFolderPath(folderPath: string): RushLibModuleType {
+  const rushLibModulePath: string = Import.resolveModule({
+    modulePath: RUSH_LIB_NAME,
+    baseFolderPath: folderPath
+  });
+
+  return _require(rushLibModulePath);
+}