JSinRust with AI

Inspiration

Instead of button, forms, menus etc, I want to control my apps through a chat conversation. AI models can call APIs, through the concept of "tool calls", which I want to leverage for this purpose. NEAR AI supports this just as good as the OpenAI models does, and I wanted to implement it for basic functions in the JSinRust project.

What it does

The JSinRust project is now equipped with tools to run javascript that it generates, which is useful for simulation. The tool runs a simple smart contract simulator in the browser that runs the WebAssembly module of the web4 smart contract. When you ask the AI to create the javascript code for a web4 page, it implements the web4_get method of the web4 contract that generates the HTML response according to what you ask it to do.

You can then test your code in the browser, before moving on to deployment.

To deploy the web4 contract javascript code, you first need an account with the web4 base Rust contract deployed. There's a tool available to the model for creating that account and deploying the contract. This is implemented through a factory contract, which is available at web4factory.near. The base contract is large, about 800Kb, and requires 9 NEAR for storage. All this is handled by the factory contract, which can be called from the AI model.

After the base contract is deployed, the keys to it is stored in the browsers localStorage, so that the AI model can do quick updates to the javascript code, which is posted using a simple function call. You may also get the keys there if you want to delete the account to release the funds deposited for storage.

How we built it

Most of the codebase is in Rust, integrating the QuickJS javascript runtime, which is compiled using Emscripten. The integration with NEAR AI is written in browser side Javascript, and there is no server in between. For the OpenAI chatgpt model, there is a proxy which is there to track costs. The user pays for ChatGPT using fungible tokens powered by the JsInRust Fungible Token contract. The code for tool call handling, simulation of generated javascript is all running in the browser. The smart contract simulator is written in pure Javascript, and mocks NEAR environment functions that it provides to the Wasm binary of the smart contract.

Challenges we ran into

The documentation for NEAR AI login did not work out of the box for using in the web browser. I will soon report the needed modifications to the docs.

Tool calls handling is complex, in order to make sure that the language model gets the correct responses in return. Having the model to create and test its own javascript is also interesting, and you can see that there sometimes multiple repeated attempts until the simulation tool confirms it to work. Writing tools that gives proper success and error responses is crucial for the model to be efficient.

Deploying the large web4 base contract of more than 800Kb is challenging because of the gas limit. Using the linkdrop contract as is will not handle it. The solution was to add an intermediate minimum self-upgrade contract into web4factory.near, that is able to receive the raw bytes of the larger contract.

Accomplishments that we're proud of

Integration of NEAR AI and ChatGPT side by side. Tool call handling that works with both of them. A contract factory that is controlled by the AI, and the tools for simulating and self testing the generated Javascript code before it is posted to the contract.

The use of Fungible tokens to pay for ChatGPT is also very useful, but this was already in the project. The user experience is slightly improved though.

What we learned

Learning that NEAR AI has powerful models, and is great for tool interaction. Experiencing the importance of proper tool error and success responses. Tool calls generating transactions must be handled with care. Providing the AI user experience to this new generation of apps is a new skill that needs to evolve.

What's next for JSinRust with AI

Streaming responses for NEAR AI. Tools to support all the contracts in JsInRust ( FT and NFT ). Popup interfaces with code editor. Integration with Ariz portfolio for creating NEAR accounting reports. Integration with WebAssembly Music for assisting in live coding of music in JavaScript.

Built With

• fermyonspin
• javascript
• nearai
• openai
• rust
• wasm32-wasip2
• web4