Exploration of cloud ide Theia plug-in system development

Time:2021-1-12

Exploration of cloud ide Theia plug-in system development

  • Theia plug-in system is very powerful, such a large project, still can maintain high quality code and clear architecture, it is worth thinking and learning
  • The definition of vscode extension API and configuration is standard and abstract

Eclipse Theia is an extensible platform, which can use the most advanced web technology to develop multi language cloud & Desktop ide.

Noun explanation

  • Theia: scalable cloud & Desktop ide platform.
  • Theia ExtensionThea is composed of a series of extensions, which provide the ability to handle widgets, commands, handlers, etcLoad at compile time
  • Theia Plugin: conceptually similar to vscode extension, one of Theia’s extensions:theia/packages/plugin-extThe loading mechanism, running environment and API are defined. Compatible with vscode extension, more powerful,Run time loading
  • VSCode Extension: Extensions loaded by vscode runtime, based on vscode extension API, conceptually similar to Theia’s plugin,Run time loading

The vscode extension can be regarded as a subset of Theia plug.

The boundary between Theia extension and plugin: extension is used for core, abstract and compile time loading; plugin is used for business, concrete and run time loading.

Thea plug type is divided into front-end and back-end (vscode only has back-end)The back end runs in an independent plug-in process; the front end runs on the web worker and communicates with the browser main process through PostMessage. The web worker here is a bit like the app service in the wechat applet architecture.

summary

Expand the ability of Theia plug to enable business parties to customize the functions and interfaces of IDE simply and flexibly.

motivation

Theia plugin’s development mode and strategyabilityandVSCode ExtensionSimilarly, it does not meet our needs:

  1. UI customization ability is very weak: the main interface only provides a small number of buttons and menus to support customization. But many practical scenarios have very strong UI requirements to meet different business capabilities. For example, the main interface of taro ide needs a lot of button menu injection and preview panels such as emulator and debugger.
  2. Configurable UI customization can’t meet customization requirements: thea plugin is based on Phosphor.js To realize the layout system, the customization ability is limited toConfigurationIn this layer, with more and more business parties in different scenarios of IDE core, it is easy to form a “configuration hell”. Therefore, while retaining the configuration, it is better to provide layout related API to let business parties use JSX to customize the layout. (refer to Kaitian)
  3. Docking with internal business and scene: such as ERP login authentication, gitlab warehouse docking, team collaboration and workspace, monitoring / operation system integration, etc. (refer to Kaitian)+Eclipse Che

Therefore, we need to expand thea’s plug-in system.

principle

  1. Shield complex concepts such as IOC / layout system / widgt, so that users can develop tide plug-ins only with vs code plug-in development experience.
  2. Reuse the design and API related to vs code extension as much as possible, and expand with reference to the existing interface or specification of vs code extension API as much as possible.
  3. Users only need to have the experience of react development to customize the layout system.

Design Overview

Design summary

  1. The plug-in system is expanded by independent extension package.

    • reference resourceseclipse-che-theia-plugin-ext, providing tide Theia plugin ext.
    • Users only need to load tide Theia plugin ext to use the extended API and configuration.
  2. Proposal referenceExtended interface and specification of vs code extensionTo expand the plug-in system from configuration, command, vs Code API and other aspects.

  3. Compared with Theia / vscode namespace, provide the tile’s namespace to access the tile API.

    • Decoupling with Theia plug

Overall design diagram

Exploration of cloud ide Theia plug-in system development

Structure of tide project

IDE core and taro ide are temporarily placed in the same project tide. It is recommended to refer to:che-theia

./
├── configs
├── examples
│   ├── browser-app
│   └── electron-app
├── extensions
│   ├── tide-theia-about
The definition of tide Theia plugin // tide API interface specification
│├ -- tide Theia plugin ext // plug in system development
│├ - tide Theia user preferences // user information related
│   ├──  ...
└──  plugins
    ├── dashboard-plugin
    ├── test-plugin
    ├── deploy-plugin
    ├── setting-plugin
    └── ...

NPM package published in@tideScope.

Vs code extension (conceptually equivalent to Theia’s plug)abilityThere are three ways to expand:

detailed design

The project will refer toExtended interface and specification of vs code extensionTo expand the plug-in system from configuration, command, vs Code API and other aspects. The most representative example of vscode expansion should beTree View APIIt has the above three ways.

Extension of distribution points configuration

Contribution Pointsyespackage.jsonincontributesField, and the plug-in registers theContribution PointsTo expand the function of vscode.

contributesConfiguration processing can be divided into scanner and handler. Mainly inplugin-extIt can be realized in the future.

scanner

plugin-ext/src/hosted/node/scanners/scanner-theia.tsInsideTheiaPluginScannerClass implements all package.json The reading method of configuration includes contribution configuration, activation events configuration, etc.

We don’t need to add a new configuration read, so we don’t need to modify it here.

handler

The final configuration handle of contribution is in thePluginContributionHandlerThe actual handler class is injected into.

// packages/plugin-ext/src/main/browser/plugin-contribution-handler.ts
export class PluginContributionHandler {
    @Inject (menuscontributionpointhandler) // inject menu related handler 
    private readonly menusContributionHandler: MenusContributionPointHandler;
    @Inject (keybindingscontributionpointhandler) // inject keybindingsrelated handler 
    private readonly keybindingsContributionHandler: KeybindingsContributionPointHandler;
    // ...

    handleContributions(clientId: string, plugin: DeployedPlugin): Disposable {
     // ...
        pushContribution('commands', () => this.registerCommands(contributions));
        pushContribution('menus', () => this.menusContributionHandler.handle(plugin));
        pushContribution('keybindings', () => this.keybindingsContributionHandler.handle(contributions));

        if (contributions.views) {
            for (const location in contributions.views) {
                for (const view of contributions.views[location]) {
                    pushContribution(`views.${view.id}`,
                        () =>  this.viewRegistry.registerView (location, view) // registration page configuration
                    );
                }
            }
        }
     // ...
    }
    registerCommandHandler(id: string, execute: CommandHandler['execute']): Disposable {}
    registerCommand(command: Command): Disposable {}
     // ...
}

expand

Different from API expansion, expansion injection point is specially reservedExtPluginApiProviderHowever, similar interfaces in Theia code are not reserved, so the following steps are adopted for expansion:

  1. definitionTidePluginContributionHandlerinheritPluginContributionHandlerclass
  2. rewritehandleContributionsmethod
  3. In containermodulerebind(TidePluginContributionHandler).to(PluginContributionHandler).inSingletonScope();

If there is a better way, please correct me.

Command development

Commands triggerTheia/VSCodeActions. Vscode code contains a lot ofbuilt-in commandsYou can use these commands to interact with the editor, control the user interface, or perform background operations.

For reference:packages/plugin-ext-vscode/src/browser/plugin-vscode-commands-contribution.ts

First, defineXXXCommandsContributionClass implementationCommandContributionAnd inject the corresponding services, such asXXXCommandsContributionPassed incommands.registerCommandFor example:

// packages/plugin-ext-vscode/src/browser/plugin-vscode-commands-contribution.ts
export class PluginVscodeCommandsContribution implements CommandContribution {
    @inject(ContextKeyService)
    protected readonly contextKeyService: ContextKeyService;
    @inject(WorkspaceService)
    protected readonly workspaceService: WorkspaceService;
    
    registerCommands(commands: CommandRegistry): void {
            commands.registerCommand ({ID: 'openinterminal'}, {// register command
            execute: (resource: URI) => this.terminalContribution.openInTerminal(new TheiaURI(resource.toString()))
        });
    }
}

Then bind to the container

bind(XXXCommandsContribution).toSelf().inSingletonScope();
bind(CommandContribution).toService(XXXCommandsContribution);

Xxxcommandscontribution will be injected into the corresponding contributionprovider, and then processed

constructor(
        @inject(ContributionProvider) @named(CommandContribution)
        protected readonly contributionProvider: ContributionProvider<CommandContribution>
) { }

Command can pass in objects as parameters, but cannot expose interfaces and components.

API development

Compared with the above two expansion methods, the expansion method of API is more complex.

Method 1: plugin ext vscode

This method is adopted by vscodePluginLifecycleInsidebackendInitPathorfrontendInitPathThese two scripts are similar to the preload script. They are preloaded before the plug-in is loaded to initialize the plug-in environment.

SpecificallyVsCodePluginScannerIn classgetLifecycle()MethodologicalbackendInitPath。 Here, backendinitpath is initialized to:backendInitPath: __dirname + '/plugin-vscode-init.js'

/**
 * This interface describes a plugin lifecycle object.
 */
export interface PluginLifecycle {
    startMethod: string;
    stopMethod: string;
    /**
     * Frontend module name, frontend plugin should expose this name.
     */
    frontendModuleName?: string; 
    /**
     * Path to the script which should do some initialization before frontend plugin is loaded.
     */
    Frontendinitpath?: string; // plug in front-end preload
    /**
     * Path to the script which should do some initialization before backend plugin is loaded.
     */
    Backendinitpath?: string; // plug in backend preload
}

And then in the pluginhostrpc classnew PluginManagerExtImpl()When invoked in an incoming init hook,initContextPassed inrequire()Method load.

be careful:initContextInsidebackendInitPathFromPluginLifecycleIt’s notExtPluginApiProvider

// packages/plugin-ext/src/hosted/node/plugin-host-rpc.ts
/**
 * Handle the RPC calls.
 */
export class PluginHostRPC {
    private apiFactory: PluginAPIFactory;

    private pluginManager: PluginManagerExtImpl;

    initialize(): void {
        this.pluginManager = this.createPluginManager(envExt, storageProxy, preferenceRegistryExt, webviewExt, this.rpc);
    }

    initContext(contextPath: string, plugin: Plugin): any {
        const { name, version } = plugin.rawModel;
        console.log('PLUGIN_HOST(' + process.pid + '): initializing(' + name + '@' + version + ' with ' + contextPath + ')');
        Const backendinit = require (contextpath); // load the backendinitpath of pluginlifecycle
        backendInit.doInitialization ( this.apiFactory , plugin); // call the doinitialization method exposed by the backendinitpath script
    }

    createPluginManager(){
        const pluginManager = new PluginManagerExtImpl({
            loadPlugin(plugin: Plugin): any {},
            async init(raw: PluginMetadata[]): Promise<[Plugin[], Plugin[]]> {
                            let backendInitPath = pluginLifecycle.backendInitPath;
                            // if no init path, try to init as regular Theia plugin
                            if (!backendInitPath) {
                                backendInitPath = __dirname + '/scanners/backend-init-theia.js';
                            }
                            self.initContext (backendinitpath, plugin); // backendinitpath from pluginlifecycle
            },
            initExtApi(extApi: ExtPluginApi[]): void {
                            const extApiInit = require( api.backendInitPath ); // load the backendinitpath injected by extpluginapiprovider
                            extApiInit.provideApi(rpc, pluginManager);
            },
            loadTests: extensionTestsPath ? async () => {}
        })
    }
}

The backendinitpath configurationplugin-vscode-init.tsThe document providesdoInitializationMethod, indoInitializationIn the method, theObject.assignMerge thea API into vscode namespace and add simple API and fields.

// packages/plugin-ext-vscode/src/node/plugin-vscode-init.ts
export const doInitialization: BackendInitializationFn = (apiFactory: PluginAPIFactory, plugin: Plugin) => {
    const vscode =  Object.assign (apifactory (plugin), {extensionkind}); // merge API 

    // use Theia plugin api instead vscode extensions
    (<any>vscode).extensions = {
        get all(): any[] {
            return vscode.plugins.all.map(p => asExtension(p));
        },
        getExtension(pluginId: string): any | undefined {
            return asExtension(vscode.plugins.getPlugin(pluginId));
        },
        get onDidChange(): theia.Event<void> {
            return vscode.plugins.onDidChange;
        }
    };
}

The essence of this method is to run the script before the plug-in is loaded, which does not involve RPCObject.assignMerge simple APIs.

This method is not as elegant as extpluginapiprovider. Some people in the community changed it to extpluginapiproviderMake “theia” and “vscode” contributed API’s #8142So far, it has not been merged.

Method 2: extpluginapiprovider

eclipse/che-theiaIt is in this way that the function is very powerful. It can be seen that:ChePluginApiProvider

The official document of Theia does not mention this method, but there is a simple introduction document under plugin ext / docThis document describes how to add new plugin api namespace

Che-Theia plug-in APIThe name space of Che is provided.

Exploration of cloud ide Theia plug-in system development

First, declare the extpluginapiprovider implementation:

// extensions/eclipse-che-theia-plugin-ext/src/node/che-plugin-api-provider.ts
export class ChePluginApiProvider implements ExtPluginApiProvider {
    provideApi(): ExtPluginApi {
        return {
            frontendExtApi: {
                initPath: '/che/api/che-api-worker-provider.js',
                initFunction: 'initializeApi',
                initVariable: 'che_api_provider'
            },
            backendInitPath: path.join('@eclipse-che/theia-plugin-ext/lib/plugin/node/che-api-node-provider.js')
        };
    }
}

Then inject it into the backend moudule:

    // extensions/eclipse-che-theia-plugin-ext/src/node/che-backend-module.ts
    bind(ChePluginApiProvider).toSelf().inSingletonScope();
    bind(Symbol.for(ExtPluginApiProvider)).toService(ChePluginApiProvider);

In this way, the front-end and back-end have the entrance of plug-in expansion.

createAPIFactory

Createapifactory is used to define the client API interface, and then mount it to the namespace of the front-end and back-end plug-ins respectively.

Creaeapifactory method, and Theia source codepackages/plugin-ext/src/plugin/plugin-context.tsThe implementation of createapifactory is consistent

export function createAPIFactory(rpc: RPCProtocol): CheApiFactory {
    return function (plugin: Plugin): typeof che {}
}

Front end entry initializeapi

Front end entry scriptche-api-worker-provider.js, implement andexport initializeApimethod. In initializeapi, pass in RPC and mount it to Che namespace.

// extensions/eclipse-che-theia-plugin-ext/src/plugin/webworker/che-api-worker-provider.ts
export const initializeApi: ExtPluginApiFrontendInitializationFn = (rpc: RPCProtocol, plugins: Map<string, Plugin>) => {
    Const cheapifactory = createapifactory (RPC); // the core is createapifactory
    const handler = {
        get: (target: any, name: string) => {
            const plugin = plugins.get(name);
            if (plugin) {
                let apiImpl = pluginsApiImpl.get(plugin.model.id);
                if (!apiImpl) {
                    apiImpl = cheApiFactory(plugin);
                    pluginsApiImpl.set(plugin.model.id, apiImpl);
                }
                return apiImpl;
            };MainPluginApiProvider
        }
        ctx['che'] = new Proxy( Object.create (null), handler); // directly mount to Che namespace
    };

Back end portal provideapi

Back end entry scriptche-api-node-provider.jsThe code needs to be exposedexport provideApi()

The back end also defines the client API interface through createapifactory.

export const provideApi: ExtPluginApiBackendInitializationFn = (rpc: RPCProtocol, pluginManager: PluginManager) => {
    cheApiFactory = createAPIFactory(rpc);
    plugins = pluginManager;

    if (!isLoadOverride) {
        overrideInternalLoad();
        isLoadOverride = true;
    }
};

And then in theoverrideInternalLoad()Methodmodule._load, makerequire('@eclipse-che/plugin')Returns the defined client API.

function overrideInternalLoad(): void {
    const module = require('module');
    // save original load method
    const internalLoad = module._load;

    // if we try to resolve che module, return the filename entry to use cache.
    module._load = function (request: string, parent: any, isMain: {}): any {
        if (request !== '@eclipse-che/plugin') {
            return internalLoad.apply(this, arguments);
        }

        apiImpl = cheApiFactory(plugin);
        return apiImpl;
    }
}

Front and back client API injection

Client API can be regarded as the definition of interface, which is exposed to the front and back runtime for plug-ins to call.

new PluginManagerExtImpl()The first parameter host passed in is the pluginhost type, in which the initextapi and other methods are implemented in the front-end and back-end respectively

export interface PluginHost {

    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    loadPlugin(plugin: Plugin): any;

    Init (data: pluginmetadata: []]: promise < [plugin []] > | [plugin []]; // initialize the plug-in

    Initextapi (extapi: extpluginapi: []): void; // initializes the front and back APIs imported from outside. Extpluginapi contains frontend extapi or backend initpath

    loadTests?(): Promise<void>;
}

Initextapi front end, hung under the window.

initExtApi(extApi: ExtPluginApi[]): void {
    if (api.frontendExtApi) {
        ctx.importScripts(api.frontendExtApi.initPath);
        ctx[api.frontendExtApi.initVariable][api.frontendExtApi.initFunction](rpc, pluginsModulesNames);
    }
}

amongconst pluginsModulesNames = new Map<string, Plugin>();A collection of plug-ins.

Initextapi backend, directly require, and run provideapi ()

initExtApi(extApi: ExtPluginApi[]): void {
    if (api.backendInitPath) {
        const extApiInit = require(api.backendInitPath);
        extApiInit.provideApi (RPC, pluginmanager); // call provideap
    }
}

Injection of server API

The server API can be regarded as the implementation of the interface, which is injected into the browser through mainpluginapiprovider to listen to the RPC messages of the client API and trigger the corresponding processing methods.

The implementation of mainpluginapiprovider should include the main (interface implementation) part of the plugin API of the new namespace.

/**
 * Implementation should contains main(Theia) part of new namespace in Plugin API.
 * [initialize](#initialize) will be called once per plugin runtime
 */
export interface MainPluginApiProvider {
    initialize(rpc: RPCProtocol, container: interfaces.Container): void;
}

Inject it into the browser’s hostedpluginsupport, and then call the initialize method injected into mainpluginapiprovider in initrpc method to initialize.

// packages/plugin-ext/src/hosted/browser/hosted-plugin.ts
@injectable()
export class HostedPluginSupport {

    @inject(ContributionProvider)
    @named(MainPluginApiProvider)
    protected readonly mainPluginApiProviders: ContributionProvider<MainPluginApiProvider>;
    
    protected initRpc(host: PluginHost, pluginId: string): RPCProtocol {
        const rpc = host === 'frontend' ? new PluginWorker().rpc : this.createServerRpc(pluginId, host); 
        setUpPluginApi(rpc,  this.container ); // initialize the server implementation of vscode API
        this.mainPluginApiProviders .getContributions().forEach(p => p.initialize(rpc,  this.container )); // initialize the interface implementation of external injection
        return rpc;
    }

}

The simplified API communication architecture is as follows:

Exploration of cloud ide Theia plug-in system development

Extpluginapiprovider is very mature, elegant and powerful. It is recommended to use this method.

Demo

See the extension / tide Theia plugin ext module of the master branch of tide project.

reference resources