Docker を使って開発しようかなと思って環境を作ってみたら、debugBasePath の設定がきかなくなっていました。
めんどくさそうなので、一度諦めましたが、やっぱり気になったのでとりあえず動くところまでやってみました。
ちなみにWebパーツしか試していません。

コンテナ起動

毎度ながら、お世話になっている DockerFile を利用させていただきます。
ありがとうございます。

https://github.com/waldekmastykarz/docker-spfx

PowerShell

docker run -it --rm --name [コンテナ名] -v [プロジェクトディレクトリ]:/usr/app/spfx -p 5432:5432 -p 4321:4321 -p 35729:35729 waldekm/spfx:1.12.1

gulp インストールと Webパーツプロジェクトの作成

Docker

spfx@5a38d4c15994:/usr/app/spfx$ npm install gulp
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies. 
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN saveError ENOENT: no such file or directory, open '/usr/app/spfx/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN enoent ENOENT: no such file or directory, open '/usr/app/spfx/package.json'
npm WARN spfx No description
npm WARN spfx No repository field.
npm WARN spfx No README data
npm WARN spfx No license field.

+ gulp@4.0.2
added 326 packages from 226 contributors and audited 327 packages in 25.004s

7 packages are looking for funding
  run `npm fund` for details

found 3 vulnerabilities (1 low, 2 moderate)
  run `npm audit fix` to fix them, or `npm audit` for details
spfx@5a38d4c15994:/usr/app/spfx$ yo @microsoft/sharepoint --package-manager npm
? ========================================================================== 
We're constantly looking for ways to make yo better! 
May we anonymously report usage statistics to improve the tool over time?    
More info: https://github.com/yeoman/insight & http://yeoman.io
========================================================================== No

     _-----_
    |       |    .--------------------------.
    |--(o)--|    |      Welcome to the      |
   `---------´   |  SharePoint Client-side  |
    ( _´U`_ )    |         Solution         |
    /___A___\    |     Generator@1.12.1     |
     |  ~  |     '--------------------------'
   __'.___.'__
 ´   `  |° ´ Y `

The npm package manager will be used.
Let's create a new SharePoint solution.
? What is your solution name? spfx
? Which baseline packages do you want to target for your component(s)? SharePoint Online only (latest)
? Where do you want to place the files? Use the current folder
Found npm version 6.14.13
? Do you want to allow the tenant admin the choice of being able to deploy the solution to all sites immediately without running any feature deployment or adding apps in sites? No
? Will the components in the solution require permissions to access web APIs that are unique and not shared with other components in the tenant? No
? Which type of client-side component to create? WebPart
Add new Web part to solution spfx.
? What is your Web part name? HelloWorld
? What is your Web part description? HelloWorld description
? Which framework would you like to use? React

   create package.json
   create config/package-solution.json
   create config/config.json
   create config/serve.json
   create gulpfile.js
   create tsconfig.json
   create .vscode/extensions.json
   create .vscode/launch.json
   create .vscode/settings.json
   create config/copy-assets.json
   create config/deploy-azure-storage.json
   create config/write-manifests.json
   create src/index.ts
   create README.md
   create .gitignore
   create tslint.json
   create src/webparts/helloWorld/components/IHelloWorldProps.ts
   create src/webparts/helloWorld/components/HelloWorld.module.scss
   create src/webparts/helloWorld/components/HelloWorld.tsx
   create src/webparts/helloWorld/HelloWorldWebPart.ts
   create src/webparts/helloWorld/loc/en-us.js
   create src/webparts/helloWorld/loc/mystrings.d.ts
   create src/webparts/helloWorld/HelloWorldWebPart.manifest.json
   create teams/e8018470-f2d3-4f37-8844-cd33966374c7_outline.png
   create teams/e8018470-f2d3-4f37-8844-cd33966374c7_color.png

npm WARN deprecated tslint@6.1.3: TSLint has been deprecated in favor of ESLint. Please see https://github.com/palantir/tslint/issues/4534 for more information.
npm WARN deprecated @types/anymatch@3.0.0: This is a stub types definition. anymatch provides its own type definitions, so you do not need this installed.
npm WARN deprecated gulp-util@3.0.8: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142       
npm WARN deprecated left-pad@1.3.0: use String.prototype.padStart()
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated fsevents@2.1.3: "Please update to latest v2.3 or v2.2"

> deasync@0.1.21 install /usr/app/spfx/node_modules/deasync
> node ./build.js

`linux-x64-node-14` exists; testing
Binary is fine; exiting

> node-sass@4.14.1 install /usr/app/spfx/node_modules/node-sass
> node scripts/install.js

Cached binary found at /home/spfx/.npm/node-sass/4.14.1/linux-x64-83_binding.node

> keytar@7.7.0 install /usr/app/spfx/node_modules/keytar
> prebuild-install || npm run build


> node-sass@4.14.1 postinstall /usr/app/spfx/node_modules/node-sass
> node scripts/build.js

Binary found at /usr/app/spfx/node_modules/node-sass/vendor/linux-x64-83/binding.node
Testing binary
Binary is fine
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^2.1.2 (node_modules/jest-haste-map/node_modules/fsevents):      
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.3.1 (node_modules/watchpack/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/@rushstack/typings-generator/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN ajv-keywords@3.5.2 requires a peer of ajv@^6.9.1 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 1955 packages from 1037 contributors and audited 2288 packages in 264.269s

97 packages are looking for funding
  run `npm fund` for details

found 218 vulnerabilities (54 low, 143 moderate, 21 high)
  run `npm audit fix` to fix them, or `npm audit` for details

      _=+#####!       
   ###########|       .-----------------------------------.
   ###/    (##|(@)    |          Congratulations!         |
   ###  ######|   \   |     Solution spfx is created.     |
   ###/   /###|   (@) |  Run gulp serve to play with it!  |
   #######  ##|   /   '-----------------------------------'
   ###     /##|(@)    
   ###########|       
      **=+####!

以下のファイルを編集

config/serve.json

hostnameを追加

{
  "$schema": "https://developer.microsoft.com/json-schemas/core-build/serve.schema.json",
  "port": 4321,
  "hostname": "0.0.0.0",
  "https": true,
  "initialPage": "https://localhost:5432/workbench",
  "api": {
    "port": 5432,
    "entryPath": "node_modules/@microsoft/sp-webpart-workbench/lib/api/"
  }
}

config/write-manifests.json

debugBasePathを追加

{
  "$schema": "https://developer.microsoft.com/json-schemas/spfx-build/write-manifests.schema.json",
  "cdnBasePath": "<!-- PATH TO CDN -->",
  "debugBasePath": "https://localhost:4321"
}

WebpackConfigurationGenerator.js

場所

\node_modules\@microsoft\spfx-heft-plugins\lib\plugins\webpackConfigurationPlugin\WebpackConfigurationGenerator.js

変更内容

Before　369行目から402行目の変更前

        let serveConfig = {};
        try {
            serveConfig = await node_core_library_1.JsonFile.loadAsync(path.join(options.folders.buildFolder, 'config', 'serve.json'));
        }
        catch (e) {
            // Do nothing.
        }
        const debugBaseUrl = `${serveConfig.https ? 'https' : 'http'}://${serveConfig.hostname || 'localhost'}:${serveConfig.port || 4321}/dist/`;
        let cdnBaseUrl;
        try {
            const writeManifestsConfig = await node_core_library_1.JsonFile.loadAsync(path.join(options.folders.buildFolder, 'config', 'write-manifests.json'));
            cdnBaseUrl = writeManifestsConfig.cdnBasePath;
        }
        catch (e) {
            // Do nothing.
        }
        config.plugins.push(new ManifestPlugin_1.ManifestPlugin({
            terminal,
            selectedLocales,
            sourceLocaleName: constants_1.DEFAULT_LOCALE,
            bundleEntries: bundleEntries,
            internalModuleBaseUrls: cdnBaseUrl ? [cdnBaseUrl] : [],
            debugInternalModuleBaseUrls: [debugBaseUrl],
            linkedExternals: linkedExternals,
            referencedProjects: referencedProjects,
            cumulativeManifestProcessor: cumulativeManifestProcessor,
            translatedStrings: localizedData,
            releaseDropPath: options.folders.releaseFolder,
            asyncOnlyDependencies: {
                asyncOnlyDependencyNames: options.configJson.asyncComponents || [],
                violationsAsErrors: options.production
            },
            componentDependenciesAuditDropPath: path.join(options.folders.releaseFolder, 'component-dependency-audit', `${path.basename(options.folders.buildFolder)}.json`)
        }));

After　369行目から404行目変更後

        let serveConfig = {};
        try {
            serveConfig = await node_core_library_1.JsonFile.loadAsync(path.join(options.folders.buildFolder, 'config', 'serve.json'));
        }
        catch (e) {
            // Do nothing.
        }
        const debugBaseUrl = `${serveConfig.https ? 'https' : 'http'}://${serveConfig.hostname || 'localhost'}:${serveConfig.port || 4321}/dist/`;
        let cdnBaseUrl;
        let debugBasePath;
        try {
            const writeManifestsConfig = await node_core_library_1.JsonFile.loadAsync(path.join(options.folders.buildFolder, 'config', 'write-manifests.json'));
            cdnBaseUrl = writeManifestsConfig.cdnBasePath;
            debugBasePath = writeManifestsConfig.debugBasePath;
        }
        catch (e) {
            // Do nothing.
        }
        config.plugins.push(new ManifestPlugin_1.ManifestPlugin({
            terminal,
            selectedLocales,
            sourceLocaleName: constants_1.DEFAULT_LOCALE,
            bundleEntries: bundleEntries,
            internalModuleBaseUrls: cdnBaseUrl ? [cdnBaseUrl] : [],
            debugInternalModuleBaseUrls: debugBasePath ? [`${debugBasePath}/dist/`] : [debugBaseUrl],
            linkedExternals: linkedExternals,
            referencedProjects: referencedProjects,
            cumulativeManifestProcessor: cumulativeManifestProcessor,
            translatedStrings: localizedData,
            releaseDropPath: options.folders.releaseFolder,
            asyncOnlyDependencies: {
                asyncOnlyDependencyNames: options.configJson.asyncComponents || [],
                violationsAsErrors: options.production
            },
            componentDependenciesAuditDropPath: path.join(options.folders.releaseFolder, 'component-dependency-audit', `${path.basename(options.folders.buildFolder)}.json`)
        }));

gulp serve (serve-deprecated) でデバッグ

とりあえず動きます。

パッケージにして展開

展開しても動きます。

感想

良い勉強になりました。
とりあえず動かせましたが、ここまでして Docker 使わなくてもいい気もします。
もっといい方法があれば教えてください。