Starport - Code Scaffolding Challenge

Inspiration

I wanted to learn more about starport and get into the world of blockchain. I am a golang developer and wanted to start with something that I already know while opening a new world of possibilities.

What it does

The existing starport CLI uses placeholders to generate new code in a project. This mechanism as per the problem was fragile and error prone. So, a new scaffolding mechanism has been implemented, where code is generated in appropriate location using code analysis for both Proto and Golang.

How we built it

To find out a new position for code generation, code analysis was used. The following pre-existing libraries to analyse the code are used:

1. Proto - github.com/jhump/protoreflect
2. Golang - go/parser

The above libraries converted code into an AST which were then searched for code locations where new code could be generated. There are a few assumptions that went in to make it all work:

1. Location of new code generation is structurally fixed. i.e.

a. The name of the function/struct where new code is generated does not change.

b. If a new code is generated in a return statement of the function, the return statement must have the same value (the arguments for the value, which is a struct declaration/function call, can change however).

1. It is expected that new code is generated in the same file where the scaffolder generated the original code previously. The user is not expected to refactor the "position" to a separate unknown file.

Challenges we ran into

There were a few challenges that were a bit tricky to solve:

1. Code generation for switch-case statements still use the placeholder mechanism because some form of structural guarantee could not be made for such placeholders. It cannot be bound to a function name and even if that assumption is made, there can be many switch-cases within that function & differentiate between the correct switch-case is error prone.
2. Code generation in the middle of the function which depends on the function environment (variables local to the function) still uses placeholder. For example: // this line is used by starport scaffolding # stargate/app/appModule in app.go.

Accomplishments that we're proud of

There were a few:

1. Making the whole scaffolding backwards compatible. If a project is created using an older starport CLI then it will still use the placeholder mechanism for it. For projects created with this new version, it will use code analysis.
2. Code generation in the middle of the function (in app.go) that did not depend on the function environment were extracted out in a separate function (func orderedInitGenesisModuleNames) in a backwards compatible way. This was possible because new code generated in this function are import variables which is readily accessible.
3. Imports and global declarations (consts, vars, funcs) do not use placeholder whatsoever for both old and new projects.
4. New code generation using code analysis can easily be added by writing up selectors, that go through the AST, by using helper funcs called wrapProtoFinder or wrapGoFinder.
5. Since, we depend on code analysis and the AST parser, if there is any code syntax error, it will be thrown out immediately to the user.
6. There are now 0 placeholders for Proto and have reduced the use of placeholders for Golang from 48 to 17.

What we learned

1. The general project structure and how it is organized.
2. Code generation that seemed like magic is actually working using plush based templates and placeholder replacements. And genny is used to create the whole code generation easier.
3. During code generation, it runs in two passes. One is dry run to check if everything is fine and second is wet run when the changes are to be committed.

What's next for Starport - Code Scaffolding Challenge

There are still a lot of placeholders that need to be replaced with "something else" in Golang. I believe the project structure may need to be changed and how it is written to make it work. But for that to happen, I will need to understand more about Cosmos SDK and how everything comes in together.

Built With

• golang