💬 Determine Extension Architecture

[jasonplatts]

Before moving too far forward with the Extension API and marketplace, we need to decide what type of environment will be used to run extensions.

JavaScript Core, https://developer.apple.com/documentation/javascriptcore, is an option. It does not support ESM, but as some have mentioned, there are ways to work around these limitations using build tools.

Initially, a Swift extension API is being developed.

However, we ultimately plan to take an approach similar to Raycast that was described by mathieudutour here, #180 (comment). The architecture involves using node and communicating with the CodeEdit application using JSON-RPC. A react-reconciler will allow extensions to create a custom UI with React components.

Thank you to everyone who contributed to the conversation! Please feel free to continue to add feedback and/or suggestions.

[underthestars-zhy]

First swift, then js. Swift is the easiest for us to implement.

[pkasila]

I would prefer using Swift for extensions, but having JavaScript API would be nice to have also, considering that the app may be ported to iPadOS later.

So, I think we need to provide Swift Extension API and prioritize it right now, but also export it to JavaScript.

[austincondiff]

I'd imagine more Javascript devs would be using this than Swift devs because they would use Xcode for Swift development for the most part correct? I think both are important and I agree with our current path.

[henryhchchc]

May I propose Lua as another option? Lua is small, portable, and fast. Actually neovim has a quite mature plugin ecosystem in Lua (see nvim-lua-guide). We can attract those plugin developers if we provide a Lua plugin interface.

[jasonplatts]

That's an interesting option @henryhchchc. I think at this point we are probably best sticking with Swift and/or JavaScript as our primary focus.

In my opinion, considering the large VS Code user base, most extensions developers are going to be looking to develop using JavaScript/TypeScript. Initially it will be important to encourage as many of those developers as possible to contribute and add features that aren't in the core application. I believe extensions are key to making CodeEdit usable for the majority of developers.

Since this is a Mac-centric application, it would be really cool to have Swift as an option, but as @austincondiff mentioned, I don't think that is what most extension developers be looking to use. However, if that what is practical to begin, I'm all for it!

Ultimately, I wish we could design this in a way that lets extension developers use whatever language they feel most comfortable. I don't want to discourage anyone from contributing. But, it probably isn't realistic without providing some form of language conversion. Additional extension language support is something we could maybe revisit further down the road?

[jasonplatts]

To the Swift developers, @pkasila, @MarcoCarnevali, @underthestars-zhy, and any others, how could we go about writing this API in Swift and then packaging/compiling/submitting the extensions in an approachable way?

Please correct me if I am wrong, VS Code uses the command line I think. Nova just takes care of everything via the GUI. There is a menu option to "Submit extension" and everything simply happens in the background. Not all developers like working in the command line, so it be nice to provide the ability for developers to build/submit the extensions from the GUI. Maybe we could require advanced builds to be performed from the command line?

[pkasila]

So, in my opinion, Swift Extension Architecture can be implemented the way described below.

Swift API

So, let's say we have a CodeEditExtensionAPI package that declares protocols to work with API (the main one is CodeEditAPIProtocol). Both the CodeEdit app and extensions link to this package. Then, the API class, conforming to CodeEditAPIProtocol, is implemented in the CodeEdit app. Then, an object of this class should be passed to the extension, so it can interact with API.

Loading extensions

All extensions are loaded using dlopen at the runtime from dynamic libraries.

Each dynamic library should have createExtension function as follows:

@_cdecl("createExtension")
public func createExtension() -> UnsafeMutableRawPointer {
    return Unmanaged.passRetained(HelloWorldExtensionBuilder()).toOpaque()
}

The API object is passed to the ExtensionBuilder.build function.

More information about plugin system using dynamic libraries in Swift can be found here

Compiling and packaging extension

Extensions should be compiled as dynamic libraries (dylibs).

The minimal extension's package structure should be as follows:

HelloWorld.ceext
| - manifest.json
| - plugin.dylib

• manifest.json stores the manifest of the extension
• plugin.dylib is the dynamic library with the extension

Submitting extensions

I think there are 2 ways:

• We simply create a Git repository for developers to submit their extensions using PRs (by providing a manifest and a link to the GitHub Release page or somewhere else as a JSON file): it will be pulled by the CodeEdit app and indexed on the device
• We create a custom catalog service (marketplace) where (and there are 2 options)
  • both extensions' manifests and packages are submitted by the developer and stored by us
  • only extensions' manifests are stored by us and packages are stored by the developer somewhere else (e.g. GitHub Releases)

Other languages

• If we are speaking about using JavaScriptCore then we can conform this API to be exported to the JavaScriptCore
• If we are speaking about any other language then there should be more or less similar way of exporting API

[jasonplatts]

Awesome @pkasila!

[jasonplatts]

Compiling and packaging extension

Extensions should be compiled as dynamic libraries (dylibs).

The minimal extension's package structure should be as follows:

HelloWorld.ceext
| - manifest.json
| - plugin.dylib

• manifest.json stores the manifest of the extension
• plugin.dylib is the dynamic library with the extension

Are we able to automate and simplify this process from the extension developers perspective through CodeEdit? Maybe a "Package and Submit Extension…" menu item?

[jasonplatts]

Submitting extensions

• both extensions' manifests and packages are submitted by the developer and stored by us

I am in favor of this option if we can make it work. I understand there will be costs involved and it probably depends on community support. I think it's the most reliable choice.

If extension developers provide their own storage, whether it be GitHub or something else, there is the possibility that these services might be down, causing some extensions to work and some other not, which could cause confusion to users. Our storage might go down too, but I would think it would likely be the entire extension library and we would at least have the ability to troubleshoot and/or explain the outage.

We also run the risk of extensions being deleted, but still appearing in the extension library.

[pkasila]

Are we able to automate and simplify this process from the extension developers perspective through CodeEdit? Maybe a "Package and Submit Extension…" menu item?

I think we can determine whether an extension project is open. Any extension's project can be a Swift Package (with a dynamic library) with manifest.json file, so if CodeEdit (actually, a specific extension for extension developers) detects that this is an extension's project, then it adds a target for the extension to be published and this target handles packaging and submitting the extension to the store.

I am in favor of this option if we can make it work. I understand there will be costs involved and it probably depends on community support. I think it's the most reliable choice.
If extension developers provide their own storage, whether it be GitHub or something else, there is the possibility that these services might be down, causing some extensions to work and some other not, which could cause confusion to users. Our storage might go down too, but I would think it would likely be the entire extension library and we would at least have the ability to troubleshoot and/or explain the outage.
We also run the risk of extensions being deleted, but still appearing in the extension library.

I think that I'll try to play around with how API for the extensions store and the store itself can be implemented a little bit. And I'll write about what I think suits our needs better later.