Codegen AST related code

[Boshen]

Background

For keeping compilation speed down for ourselves and our downstream crate users, I have avoided using macros to generate boilerplate code at all cost.

All performance issues (runtime and compilation speed) are considered as bugs in this project.

But due to the amount of AST related supporting code has expanded over the last few months, and our desire to experiment with different ideas with the AST, our current approach of manually updating everything is no longer functional.

The goal of this task is to provide code generation for all AST node related code.

Requirements

• no build.rs published to the user
• all generated code are checked into git
• no nightly
• Rust code residing in crates/oxc_ast should be the source of truth

Workflow

• a user changes the oxc_ast crate
• a watch change picks it up
• parse all #[visited_node]
• saves them into a model, e.g. struct ASTModel { name: String, attributes: Vec<Attributes> }
• generate code and save file

Code to generate

• ASTKind (Milestone 1)
• GetSpan (Milestone 2)
• Hash, PartialEq implementations (Milestone 3)
• ASTBuilder (Milestone 4)
• Visit (Milestone 5)
• VisitMut (Milestone 5)
• Traverse, remove current js code (Milestone 6)

• typescript definition for napi and wasm moving the scope to AST transfer.

Every milestone should be a stack of PRs, since they will involve changing other files as well.

[overlookmotel]

The AST also includes some types which are in oxc_syntax e.g. AssignmentOperator, BinaryOperator etc. Should we move those types into oxc_ast to simplify which files the codegen needs to read?

(I've never been clear why these types are in oxc_syntax, since they are part of AST)

[Boshen]

Should we move

No, let's not scope creep.

There are other traits and functions that depend on these common operators.

[Boshen]

Are we able to manually inline @inherit MemberExpression and generate all the other utilities for these inherit_variants! enums?

[rzvxa]

Are we able to manually inline @inherit MemberExpression and generate all the other utilities for these inherit_variants! enums?

Can you elaborate on it? Wouldn't we lose some information if we don't express the inheritance in the type system? While the actual code would behave the same, It might be useful for the user to see this as the definition:

inherit_variants! {
/// Argument
///
/// Inherits variants from [`Expression`]. See [`ast` module docs] for explanation of inheritance.
///
/// [`ast` module docs]: `super`
#[visited_node]
#[repr(C, u8)]
#[derive(Debug, Hash)]
#[cfg_attr(feature = "serialize", derive(Serialize, Tsify))]
#[cfg_attr(feature = "serialize", serde(untagged))]
pub enum Argument<'a> {
    SpreadElement(Box<'a, SpreadElement<'a>>) = 64,
    // `Expression` variants added here by `inherit_variants!` macro
    @inherit Expression
}
}

Instead of this:

/// Argument
///
/// Inherits variants from [`Expression`]. See [`ast` module docs] for explanation of inheritance.
///
/// [`ast` module docs]: `super`
#[visited_node]
#[repr(C, u8)]
#[derive(Debug, Hash)]
#[cfg_attr(feature = "serialize", derive(Serialize, Tsify))]
#[cfg_attr(feature = "serialize", serde(untagged))]
pub enum Argument<'a> {
    SpreadElement(Box<'a, SpreadElement<'a>>) = 64,

    BooleanLiteral(Box<'a, BooleanLiteral>) = 0,
    NullLiteral(Box<'a, NullLiteral>) = 1,
    NumericLiteral(Box<'a, NumericLiteral<'a>>) = 2,
    BigIntLiteral(Box<'a, BigIntLiteral<'a>>) = 3,
    RegExpLiteral(Box<'a, RegExpLiteral<'a>>) = 4,
    StringLiteral(Box<'a, StringLiteral<'a>>) = 5,
    TemplateLiteral(Box<'a, TemplateLiteral<'a>>) = 6,

    Identifier(Box<'a, IdentifierReference<'a>>) = 7,

    MetaProperty(Box<'a, MetaProperty<'a>>) = 8,
    Super(Box<'a, Super>) = 9,

    ArrayExpression(Box<'a, ArrayExpression<'a>>) = 10,
    ArrowFunctionExpression(Box<'a, ArrowFunctionExpression<'a>>) = 11,
    AssignmentExpression(Box<'a, AssignmentExpression<'a>>) = 12,
    AwaitExpression(Box<'a, AwaitExpression<'a>>) = 13,
    BinaryExpression(Box<'a, BinaryExpression<'a>>) = 14,
    CallExpression(Box<'a, CallExpression<'a>>) = 15,
    ChainExpression(Box<'a, ChainExpression<'a>>) = 16,
    // .
    // .
    // .
}

---

On another note, if we get the macro caching to work (related to this) we can have this without any compilation penalty which I think is much superior(And I guess @overlookmotel would agree).

/// Argument
///
/// Inherits variants from [`Expression`]. See [`ast` module docs] for explanation of inheritance.
///
/// [`ast` module docs]: `super`
#[inheritance]
#[visited_node]
#[repr(C, u8)]
#[derive(Debug, Hash)]
#[cfg_attr(feature = "serialize", derive(Serialize, Tsify))]
#[cfg_attr(feature = "serialize", serde(untagged))]
pub enum Argument<'a> {
    SpreadElement(Box<'a, SpreadElement<'a>>) = 64,
    #[inherit] // or maybe `#[inline]` since it is more rust-like?
    Expression(Expression<'a>),
}

[rzvxa]

If we want to have the inheritance expanded we should do the expansion with the codegen to keep it consistent and schema close to the ECMAspec. But IMHO, manually inlining the variants is going to be a maintenance nightmare. because they would easily drift apart with some time and careless reviewing(which always happens no matter how many people review the same piece of code).