# Get Started (/docs) For any Questions you may have about mist, [visit the FAQ here](/faq). ## 1. Installation [#1-installation] Mist is currently distributed as a Cargo crate. To get started, you'll need to have the Rust toolchain installed. Run the following command to install the Mist toolchain: ```bash title="Terminal" cargo install mist-lang@0.4.0 ``` > (ℹ) For a more streamlined development experience, Install the following VSCode extensions: > > 1. [Mist syntax highlighting VSCode extension](https://marketplace.visualstudio.com/items?itemName=selimaj-dev.mist-syntax) > 2. [Mist analyzer LSP VSCode extension](https://marketplace.visualstudio.com/items?itemName=selimaj-dev.mist-analyzer) Once the installation finishes, verify it by checking the version: ```bash title="Terminal" mist version ``` *** ## 2. Setting Up Your Mist Project [#2-setting-up-your-mist-project] ```bash mist new mist-hello-world ``` or ```bash mist init ``` ```mist title="src/main.mist" void main() { println!("Hello World!"); } ``` ### 3. Build and Run [#3-build-and-run] ```bash title="Terminal" mist run ``` *** ## Command Reference [#command-reference] | Command | Alias | Description | | | -------------- | ----- | ------------------------------------------ | ----------------- | | mist run | r | | run the project | | mist build | b | | build the project | | mist transpile | t | transpile the project | | | mist check | c | check the project | | | mist test | | test the project | | | mist publish | | publish the project | | | mist bench | | benchmark the project | | | mist doc | | build documentation | | | mist fix | | automatically fix warnings | | | mist clippy | | lint the project | | | mist clean | | clean build artifacts | | | mist init | | initialize a project in the current folder | | | mist new | | create a new project | | | mist version | -v | print the compiler version | | | mist help | -h | print this message | | # Status & Limitations (/docs/limitations) Mist is currently in a **Beta** stage (latest: v0.4.0). Our current priority is exploring **Syntax and Features**. We believe in stabilizing the developer experience and the "feel" of the language before locking in the deep architectural logic of the compiler. The compiler, transpiler, and CLI are partially bootstrapped — written in Mist itself. This gives us real-world feedback on every language design decision. You may encounter weird illogical syntax errors, please report them at [https://github.com/mist-go/mist/issues](https://github.com/mist-go/mist/issues) with the context. # Philosophy (/docs/philosophy) **Mist** is a pragmatic systems programming language built on Rust. It uses C/C++ style syntax for quick onboarding while compiling directly to efficient Rust code with zero-cost abstractions and no runtime overhead. All of your favorite Rust libraries work with Mist. It is designed for **low-level applications and embedded programming**, where a type-first approach is preferred and staying close to the metal matters. *** ## C++ You Know, Rust You Trust [#c-you-know-rust-you-trust] Mist is meant to feel like C++ but **be** Rust. It bridges the gap between the ergonomics of a classic systems language and the safety guarantees of Rust's ownership model. ## Built for Low-Level and Embedded [#built-for-low-level-and-embedded] Mist is designed from the ground up for systems where resources are constrained and control is required: * **No runtime** — no garbage collector, no hidden allocator, no VM. * **Zero-cost abstractions** — classes, traits, and generics all resolve at compile time. * **Works with existing Cargo tooling** — use any Rust crate as a library. *** ## A Transparent Surface Layer [#a-transparent-surface-layer] Mist is not a replacement for Rust; it is an **ergonomic interface** for it. It provides a way to interact with the world's most powerful systems model through a cleaner lens. * **Zero-Cost Abstractions:** Every high-level structure in Mist — classes, enums, traits — maps directly to optimized Rust primitives. No magic, no hidden runtime. * **Grounded Safety:** Mist doesn't hide memory safety; it makes it easier to express. You retain the full power of the borrow checker with syntax that feels like classic C++. * **Native Fluency:** Mist compiles to idiomatic, readable Rust, making it a first-class citizen of the ecosystem. Use any crate while writing code that feels distinctly Mist. *** ## Bootstrapped Development [#bootstrapped-development] Mist is partially bootstrapped — the compiler's CLI and transpiler modules are written in Mist itself. This gives us first-hand experience with the language's ergonomics and drives real-world improvements with every release. ## Why Mist? [#why-mist] * **Familiarity:** C/C++ developers can be productive immediately — no new optical model to learn (Note: Rust's semantics still need to be learned). * **Safety:** Rust's borrow checker and type system under every expression, no exceptions. * **Control:** Explicit types, explicit pointers, explicit allocation — nothing hidden. * **Ecosystem:** The entire Rust crate ecosystem available at your fingertips. # Compiler Architecture (/docs/guide/architecture) ### Pipeline Overview [#pipeline-overview] The Mist compiler operates in distinct phases: ``` Source (.mist) │ ▼ ┌─────────────┐ │ Lexer │ (pest PEG grammar) │ & Parser │ └─────────────┘ │ AST ▼ ┌─────────────┐ │ Semantics │ (field init checking) └─────────────┘ │ ▼ ┌─────────────┐ │ Codegen │ (AST → Rust source) └─────────────┘ │ .rs + .map.json ▼ ┌─────────────┐ │ cargo │ (Rust compilation) └─────────────┘ │ ▼ Binary / Library ``` ### Crate Organization [#crate-organization] The compiler is written in Rust and Mist itself, organized into four main crates: | Crate | Language | Role | | ----------------- | -------- | ----------------------------------------------------------------------------------- | | **mist-parser** | Rust | PEG parsing via pest, AST construction, semantic checks, position mapping | | **mist-codegen** | Rust | AST → Rust source code generation | | **mist-analyzer** | Rust | Language server bridging mist-editor ↔ rust-analyzer | | **mist-api** | Mist | Orchestrates transpilation, module tree building, cargo invocation, error remapping | # Attributes (/docs/guide/attributes) Inner attributes apply to the containing module: ```mist #![allow(unused_variables)] ``` Outer attributes apply to the next item: ```mist #[derive(Debug, Clone)] struct Point { i32 x; i32 y; } #[test] void my_test() { assert_eq!(1, 1); } ``` Attribute syntax: * `#[path]` * `#[path = literal]` * `#[path(item1, item2, ...)]` # Classes (/docs/guide/classes) Mist introduces `class` as syntactic sugar for a Rust struct with a virtual method table (vtable). ```mist pub class Animal { str& name; pub constructor(str& name) { self.name = name; } pub void speak(&self) { println!("..."); } } ``` Class fields can have default initializers: ```mist class Player { i32 health = 100; str& name; } ``` ### Virtual Methods [#virtual-methods] The `virtual` keyword marks methods as dispatchable through the vtable, enabling polymorphic behavior: ```mist pub class Animal { str& name; pub constructor(str& name) { self.name = name; } pub virtual void speak(&self) { println!("..."); } } ``` When a method is marked `virtual`, it can be overridden in subclasses and will be dispatched dynamically at runtime through the vtable. Non-virtual methods are called statically. ### Inheritance [#inheritance] ```mist class Dog : Animal { pub constructor(str& name) { super = Super::new(name); } void speak(&self) override { println!("Woof!"); } } ``` The `override` keyword supports explicit base class targeting (Required if overriding a nested parent): ```mist void speak(&self) override(Animal) { println!("Woof!"); } ``` Methods marked `override` are automatically virtual if the parent method is virtual. The semantic analyzer validates that overridden methods match the parent's signature. ### Implementations [#implementations] You can implement directly on the class body: ```mist class Circle { constructor() {} impl std::fmt::Display { Result<(), std::fmt::Error> fmt(&self, Formatter<'_> f) { write!(f, "●") } } } ``` *** For details on how classes are compiled — vtable layout, constructor codegen, and safety verification — see [Class Internals](/docs/internals/classes). # Control Flow (/docs/guide/control-flow) ### If/Else [#ifelse] ```mist if condition { // body } else if other_condition { // body } else { // body } ``` Used as an expression: ```mist let x = if true { 1 } else { 2 }; ``` ### While [#while] ```mist while condition { // body } ``` ### For [#for] ```mist for pattern in iterator { // body } for i in 0 .. 10 { // body } ``` ### C-Style For [#c-style-for] ```mist for (let mut i = 0; i < 10; i++) { // body } ``` ### Loop [#loop] ```mist loop { // infinite loop } ``` ### Match [#match] ```mist match value { pattern1 => expr, pattern2 => { // block body } pattern3 | pattern4 => expr, } ``` ### Break / Continue [#break--continue] ```mist break; continue; ``` ### Return [#return] ```mist return; return value; ``` # Functions (/docs/guide/functions) ```mist // Return type is `void` (unit) void greet(str& name) { println!("Hello, {name}"); } // With return type i32 add(i32 a, i32 b) { return a + b; } // Expression body (last expression is the return value) i32 square(i32 x) { x * x } // Public function pub i32 multiply(i32 a, i32 b) { a * b } // Generic function T identity(T value) { value } // Unsafe function unsafe i32 dangerous() { 42 } ``` ### Self Parameter [#self-parameter] Methods can take `self`, `mut self`, `&self`, `&mut self`, and `&'a self` / `&'a mut self` with lifetimes: ```mist pub void set_value(&mut self, i32 v) { self.value = v; } ``` # Generics (/docs/guide/generics) ```mist // Generic function T id(T x) { x } // Generic struct struct Pair { A first, B second, } // Generic enum enum Result { Ok(T); Err(E); } // Generic with trait bounds T max(T a, T b) { if a > b { a } else { b } } // Lifetime generics void process<'a>(&'a str& data) { // ... } ``` Generic syntax uses `<` `>` delimiters. Lifetimes are prefixed with `'`. # Modules & Imports (/docs/guide/modules) Mist's module system was created to fix Rust's dreaded parent-declared module system, files declare themselves and their publicity via `pub module my_mod;` as the first statement. ```mist // Declare a module pub module foo; // Import use std::collections::HashMap; // Re-exporting import (alias) pub use my_module::MyType; ``` ### Include Directives [#include-directives] Mist supports C-style include directives for ```mist // Global include (std::Display) // Includes all of the items *inside* the path #include // Use include (std::Display) // Includes *the path* itelf to the root. #use std::fmt::Display; // Local include // Textual include, transpiles to rust include! macro via .rs extension #include "my_local_module.mist" ``` Include directives allow you to bring in external code without using the module system, which is useful for C interop and local file organization. ### Module Resolution [#module-resolution] Mist maps the module tree to Rust's module system: | Mist Path | Rust Output | | ------------------------ | ------------------------ | | `src/main.mist` | `.mist/src/main.rs` | | `src/utils/package.mist` | `.mist/src/utils/mod.rs` | | `pub module foo;` | `.mist/src/foo.rs` | A `package.mist` file acts as a directory's module root, analogous to `mod.rs`, but it's not required, as long as the directory contains `.mist` files, it will auto generate `mod.rs`. #### *💡 Tip*: File names don't matter, `module my_mod` determines the name. [#-tip-file-names-dont-matter-module-my_mod-determines-the-name] # Operators, Macros & Closures (/docs/guide/operators) ### Binary Operators [#binary-operators] | Operator | Description | | -------- | ------------------ | | `+` | Addition | | `-` | Subtraction | | `*` | Multiplication | | `/` | Division | | `%` | Modulus | | `==` | Equality | | `!=` | Inequality | | `<` | Less than | | `>` | Greater than | | `<=` | Less or equal | | `>=` | Greater or equal | | `&&` | Logical AND | | `\|\|` | Logical OR | | `&` | Bitwise AND | | `\|` | Bitwise OR | | `^` | Bitwise XOR | | `<<` | Left shift | | `>>` | Right shift | | `=` | Assignment | | `+=` | Add assign | | `-=` | Subtract assign | | `*=` | Multiply assign | | `/=` | Divide assign | | `%=` | Modulus assign | | `&=` | Bitwise AND assign | | `\|=` | Bitwise OR assign | | `^=` | Bitwise XOR assign | | `<<=` | Left shift assign | | `>>=` | Right shift assign | | `..` | Range (exclusive) | | `..=` | Range (inclusive) | | `->` | Pointer write | ### Prefix Operators [#prefix-operators] | Operator | Description | | -------- | ----------------- | | `*` | Dereference | | `&` | Reference | | `&mut` | Mutable reference | | `!` | Logical NOT | | `-` | Numeric negation | ### Postfix Operators [#postfix-operators] | Operator | Description | | ---------- | -------------------- | | `.field` | Field access | | `.0` | Tuple field access | | `()` | Function call | | `[]` | Index | | `{ f: v }` | Struct literal | | `as Type` | Type cast | | `?` | Try (error prop) | | `++` | Increment | | `--` | Decrement | | `!()` | Macro call (paren) | | `![]` | Macro call (bracket) | | `!{}` | Macro call (brace) | ### Macros [#macros] Mist reuses Rust's macro system directly: ```mist println!("hello"); assert_eq!(a, b); vec![1, 2, 3]; ``` Macro calls use `!` followed by parentheses, brackets, or braces. ### Closures [#closures] ```mist let add = (a, b) => a + b; let result = add(2, 3); // 5 f64 fn() square = (x) => x * x; ``` Closures can have explicit return types: ```mist let transform = i32 (x) => { x * 2 }; ``` # Patterns (/docs/guide/patterns) ```mist // Literal patterns match x { 1 => "one", 2 => "two", _ => "other", } // Tuple patterns let (a, b) = (1, 2); // Struct patterns match value { Point { x, y } => x + y, Point { x: 0, y } => y, _ => 0, } // Named tuple patterns (newtype) let MyType(value) = my_var; // Wildcard / rest let _ = get_side_effect(); match x { 1 => expr, .. => expr, // rest / etc } // Mutable binding in pattern let mut x = 42; match ref_to_option { Some(mut value) => value += 1, None => {}, } ``` # Structs & Enums (/docs/guide/structs-enums) ### Structs [#structs] ```mist pub struct Point { i32 x; i32 y; } pub struct Generic { T value; } ``` Fields can be public or private: ```mist struct User { pub str& name; i32 age; // private } ``` ### Enums [#enums] ```mist pub enum Option { Some(T); None; } pub enum Message { Quit; Move(i32, i32); Write { str& content; i32 length; }; } ``` Enum variants can be: * **Named** — `Variant` * **Tuple** — `Variant(T1, T2)` * **Struct** — `Variant { T1 field1; T2 field2; }` # Syntax Overview (/docs/guide/syntax-overview) ### Comments [#comments] ```mist // Line comments only ``` ### Literals [#literals] ```mist 42 // Integer 3.14 // Float true // Boolean false // Boolean "hello" // String (1, true, "x") // Tuple [1, 2, 3] // Array ``` ### Identifiers & Keywords [#identifiers--keywords] Keywords are reserved and cannot be used as identifiers: `if`, `else`, `fn`, `for`, `while`, `match`, `return`, `break`, `continue`, `struct`, `enum`, `class`, `trait`, `impl`, `use`, `pub`, `mut`, `let`, `true`, `false`, `dyn`, `loop`, `unsafe`, `override`, `const`, `type`, `virtual` Identifiers follow the pattern `[a-zA-Z_][a-zA-Z0-9_]*`. ### Visibility [#visibility] ```mist // Private (default) void internal() { } // Public pub void external() { } // Public to specific path pub(crate) void crate_only() { } pub(super) void parent_only() { } pub(my::module) void module_only() { } ``` # Traits & Impls (/docs/guide/traits-impls) ### Traits [#traits] ```mist pub trait Drawable { void draw(&self); } pub trait Comparable : Eq { i32 cmp(&self, T other); } ``` Trait requirements are specified after `:`: ```mist trait MyTrait : SuperTrait + OtherTrait { void required_method(&self); void another(&self); } ``` ### Impl Blocks [#impl-blocks] ```mist impl MyType { void method(&self) { } } impl Trait for MyType { void method(&self) { } } impl GenericTrait for MyType { void method(&self, T value) { } } ``` # Types (/docs/guide/types) ### Type System [#type-system] ```mist i32 // Path type str& // Reference type i32 mut& // Mutable reference i32 'a& // Reference with lifetime i32 unsafe& // Const pointer i32 mut unsafe& // Mutable pointer (i32, bool) // Tuple type i32[]& // Array type (static non-fixed size) i32[10] // Array type (fixed size) bool fn(i32) // Function pointer type bool Fn(i32) // Closure trait (Fn) bool FnMut(i32) // Closure trait (FnMut) bool FnOnce(i32) // Closure trait (FnOnce) dyn Trait // Trait object void // Unit type (maps to Rust's ()) 'lifetime // Lifetime ``` Type expressions can be composed: ```mist type_expr = { (void | path_type | tuple_type | dyn_type) ~ (unsafe_ref_type | ref_type | fn_type)* } ``` This means types are written left-to-right naturally: ```mist i32& // &i32 i32 mut& // &mut i32 i32 'a& // &'a i32 bool fn(i32) // fn(i32) -> bool ``` ### Type Aliases [#type-aliases] ```mist type MyInt = i32; type Result = std::result::Result; ``` # Variables (/docs/guide/variables) ```mist let x = 42; // Type-inferred immutable let mut y = 10; // Mutable variable i32 z = 100; // Explicit type annotation str& s = "hello"; // Typed string reference bool b = true; // Typed boolean f64 f = 3.14; // Typed float (i32, str&) as x, y = (1, "hello"); // Typed tuple destructuring let (a, b) = (1, "two"); // Destructuring let (x, (y, z)) = (1, (2, 3)); // Nested destructuring ``` Variable declarations follow either of two forms: ```mist let [= ]; [= ]; ``` ### Tuple Variable Declarations [#tuple-variable-declarations] Tuple variables can be declared with explicit type annotations using `as`: ```mist (i32, bool) as x, y = (1, true); i32 a, b = (1, 2, 3); ``` ### Const and Static [#const-and-static] ```mist const MAX: i32 = 100; static NAME: str& = "hello"; ``` # Builder & Error Remapping (/docs/internals/builder) The builder module (`builder.mist` in the `mist_api` crate) wraps `cargo` to: 1. Spawn `cargo` with `--message-format=json` 2. Parse JSON compiler messages 3. For each diagnostic span, look up the corresponding `.map.json` file 4. Remap Rust line/column to Mist line/column using the mapping 5. Display errors/warnings with Mist source locations and context lines ```rust pub fn build(args: Vec, root: PathBuf) -> bool { // Spawn cargo with JSON output // Parse CompilerMessage for each span // Look up rev_mapper::Mapping from .map.json // Remap to Mist positions // Print diagnostics } ``` ### Diagnostic Output [#diagnostic-output] Errors and warnings are displayed with Mist source context: ``` src/main.mist:10:5 Error: mismatched types let x: i32 = "hello"; ^^^^^^^ ``` # Class Internals (/docs/internals/classes) This page covers the implementation details of Mist classes. You don't need to read this to use classes — it's here for curiosity and compiler contributors. ### Under the Hood [#under-the-hood] A class `Dog : Animal` generates: 1. A Rust struct with a `_super: Animal` field (or `_vptr: &'static [*const c_void]` for root classes) 2. A vtable constant with function pointers for each public method 3. An `impl` block with `Deref` and `DerefMut` 4. Method trampolines (`__m_`) that are dispatched through the vtable 5. A `new()` constructor that initializes via `MaybeUninit` and calls the user's `constructor(&mut self)` The vtable is unified: parent entries are copied, overridden entries replace parent slots, and new methods are appended. ### Safety [#safety] Classes use `MaybeUninit::zeroed().assume_init()` inside the generated `new()` function — an inherently unsafe operation. This raises a natural question: **Are class constructors unsafe?** The answer is **no**. The compiler statically verifies that every field is initialized before the constructor returns, eliminating the undefined behavior that raw `MaybeUninit` would normally carry. #### Static Field Initialization Verification [#static-field-initialization-verification] When a class has a constructor, the semantic checker (`check_class_semantics`) collects every declared field and walks the constructor body to prove each one is written to: 1. **Direct assignment tracking** — Expressions like `self.field = value` are recognized as mutations of `field`. The analyzer checks for `=` and `->` operators whose left-hand side is a `self.field` path. 2. **`&mut self.field` tracking** — Taking a mutable reference to a field (`&mut self.field`) also counts as initializing it, since the reference can only be taken if the field is being set up. 3. **Transitive method calls** — If the constructor calls `self.helper()`, the analyzer follows into `helper`'s body and tracks which fields *it* initializes. This transitively propagates through nested calls: ```mist class Player { str& name; i32 health; pub constructor(str& name) { self.name = name; self.setup_health(); } void setup_health(&mut self) { self.health = 100; // Counts toward constructor's verification } } ``` 4. **Branch intersection** — For `if`/`else`, `match`, and loops, fields must be initialized in **all** branches. If one branch initializes `x` but another does not, `x` is considered uninitialized. This ensures soundness regardless of the runtime path: ```mist pub constructor(bool flag) { if flag { self.health = 100; } else { self.health = 0; } // Both branches init health ✓ } ``` 5. **Super initialization** — When a class inherits, the `_super` field is added to the required-field list. Any assignment to `super` counts: ```mist pub constructor(str& name) { super = Super::new(name); } ``` If any field is uninitialized after the full analysis, a compile-time error is reported with the field's exact source location: ``` class field `Player.health` is uninitialized ``` #### Override Validation at Compile Time [#override-validation-at-compile-time] When a method uses the `override` keyword, the codegen emits a hidden test function that verifies the Deref chain at compile time: ```rust #[allow(invalid_value)] fn __test_vt() { let this: &Self = &unsafe { std::mem::MaybeUninit::::zeroed().assume_init() }; let _: &Target = this; // Forces compiler to check Deref } ``` This ensures `&Self` can always deref into the base class type. If the inheritance hierarchy is invalid, the Rust compiler rejects it. #### VTable Safety [#vtable-safety] The `_vptr` (vtable pointer) is set *twice* during construction: 1. **Before** the constructor body runs — enabling virtual dispatch inside the constructor itself 2. **After** the constructor body — in case a base-class constructor ran and overwrote the pointer This ensures that virtual method calls work correctly even during object construction, without exposing uninitialized memory through the vtable. #### Summary [#summary] | Risk | Mitigation | | -------------------------------------- | ------------------------------------------------------------------------ | | Uninitialized fields via `MaybeUninit` | Static field-initialization verification rejects incomplete constructors | | UB from reading uninitialized fields | Intersection analysis ensures all branches init the same fields | | Invalid override signatures | Compile-time Deref test validates the inheritance chain | | Vtable corruption during construction | `_vptr` is set before and after the constructor body | | Unsafe code in generated constructors | `#[allow(invalid_value)]` is scoped to the generated `new()` only | The `MaybeUninit` pattern is an implementation detail of the generated code — the Mist compiler proves soundness at the language level, so the user's constructor body is safe Mist code with no manual `unsafe` annotations required. # Code Generation (/docs/internals/codegen) The code generator converts the Mist AST into Rust source code directly — no intermediate representation. ### Key files [#key-files] * `src/lib.rs` — `RustCodegen` struct with output buffer, indentation tracking, `Mapping` for position translation * `src/top_level.rs` — Generates Rust for top-level items (structs, enums, traits, functions, impls, imports, type aliases, const/static) * `src/statement.rs` — Generates Rust for statements and blocks * `src/expr.rs` — Generates Rust for expressions (literals, paths, binary ops, closures, arrays, prefix/postfix) * `src/class_decl.rs` — Class-specific code generation (struct + vtable + impl blocks + Deref) ### Codegen Design [#codegen-design] The `RustCodegen` maintains: * A string `output` buffer * An `indent` level (4 spaces per level) * A `Mapping` that records `(RustMap, MistMap)` pairs for source position remapping * A current `position` tracker (`RustMap(line, column)`) Generation follows the `GenRust` trait: ```rust pub trait GenRust { fn gen_rust(&self, ctx: &mut Context, cg: &mut RustCodegen); } ``` And `GetRust` for simple string-returning types: ```rust pub trait GetRust { fn get_rust(&self) -> String; } ``` The `Context` carries optional expression path information for class `super` / `Super` resolution. ### Class Codegen [#class-codegen] Classes are the most complex codegen path. `ClassProcessedData` analyzes a class declaration and emits: 1. **Struct declaration** — `struct ClassName { pub _super: Parent, pub field1: T1, ... }` (or `_vptr` for root classes) 2. **Vtable constants** — Index constants `__FN_METHOD` and a `__V_TABLE` static array of function pointers. For inherited classes, parent vtable entries are copied and overridden entries replaced. 3. **Constructor** — `pub fn new(...) -> Self` that creates an uninitialized instance via `MaybeUninit`, sets the vtable pointer, writes field defaults, calls `self.constructor(...)`, and returns `this` 4. **Method trampolines** — Public methods with `self` get wrapper functions `__m_method` stored in the vtable, plus virtual dispatch methods 5. **Override support** — Methods marked `override` are validated at compile time via generated test code 6. **Deref impls** — `impl Deref for Child` and `DerefMut` for inherited classes 7. **Impl declarations** — Inner `impl` blocks are rewritten to use the self type # LSP Support (/docs/internals/lsp) The Mist Language Server Protocol implementation runs a headless `rust-analyzer` instance and bridges Mist editor requests to Rust positions. ### Architecture [#architecture] ``` Editor (LSP client) │ ▼ ┌─────────────────────┐ │ mist-analyzer │ │ (Rust + Mist) │ └─────────────────────┘ │ ▲ │ JSON-RPC │ ▼ │ ┌─────────────────────┐ │ rust-analyzer │ │ (headless child) │ └─────────────────────┘ ``` ### Flow [#flow] 1. Editor sends Mist file edits to `mist-analyzer` via LSP 2. `mist-analyzer` transpiles Mist to Rust 3. The transpiled Rust is forwarded to the headless `rust-analyzer` process 4. For goto-definition, hover, and completion, a unique marker token is injected at the cursor position 5. The transpiled output with the marker is sent to `rust-analyzer` 6. The marker position is located in the response 7. Results are mapped back to Mist positions using the `rev_mapper` ### Key Features [#key-features] * **Full document sync** — Open, change, save, close * **Go to definition** — Maps through transpiled Rust positions * **Completions** — Supports trigger characters `:`, `.`, `'`, `(` * **Hover** — Type information and docs * **Diagnostics** — Real-time errors from transpilation failures and Rust compilation * **Formatting** — Document formatting support * **Auto-import** — Automatically inserts `pub module ;` when new `.mist` files are created * **File watching** — Monitors `**/*.mist` for new files ### Diagnostic Remapping [#diagnostic-remapping] When a transpilation error occurs in the `mist-analyzer`: 1. Parse errors are converted to `Diagnostic` with Mist source positions 2. Semantic errors (uninitialized class fields) use the stored line/column 3. Rust compiler errors are remapped via the `Mapping` system back to Mist positions # Parsing (/docs/internals/parsing) The parser is built with [pest](https://pest.rs), a PEG parser generator. Grammar rules are defined in `grammar.pest` (693 rules). ### Key files [#key-files] * `grammar.pest` — PEG grammar defining the entire language syntax * `src/parser/common/` — Parse rule → AST conversions for expressions, statements, types, declarations * `src/parser/items/` — Parse rule → AST conversions for top-level items (structs, enums, classes, functions, traits, impls, attributes) * `src/ast/` — AST node types (expr.rs, statement.rs, top\_level.rs) * `src/error.rs` — Error types (PreAst parsing errors, Ast generation errors) ### Parsing entry points [#parsing-entry-points] ```rust // Parse a complete program pub fn parse<'a>(source: &'a str) -> Result> // Parse only the module declaration pub fn parse_module<'a>(source: &'a str) -> Result, ParseError<'a>> ``` ### Grammar Structure [#grammar-structure] The grammar follows a layered approach: 1. **Lexical rules** (silent) — `WHITESPACE`, `COMMENT`, `identifier`, `integer`, `float`, `string_lit` 2. **Primary expressions** — literals, paths, tuples, arrays, closures, control flow 3. **Term** — prefix operators + primary + postfix operators 4. **Expression** — terms joined by binary operators (via Pratt parser) 5. **Statements** — variable declarations, control flow (`if`, `while`, `for`, `match`, `loop`), blocks 6. **Top-level items** — functions, structs, enums, classes, traits, impls, type aliases, imports, module declarations, constants ### AST Structure [#ast-structure] The AST preserves source positions via `Spanned`: ```rust pub struct Spanned { pub line: usize, pub column: usize, pub item: T, } ``` Top-level items are wrapped as: ```rust pub struct TopLevel(pub Spanned, pub Vec); ``` Expressions use a fix-point representation for prefix/postfix operators: ```rust Expression::Fix { initial: Box, prefixes: Vec, postfixes: Vec, } ``` Binary expressions use Pratt parsing for correct precedence: ```rust Expression::Binary { lhs: Box, op: String, rhs: Box, } ``` All operators are left-associative with a single precedence level. # Position Mapping (/docs/internals/position-mapping) Mist maintains a bidirectional mapping between Mist source positions and Rust output positions. This is essential for: 1. **Error remapping** — Rust compiler errors point to the correct Mist source location 2. **LSP features** — Go-to-definition, hover, and completion work from Mist source ### Data Structure [#data-structure] The mapping is stored as pairs: ```rust pub struct Mapping { pub mist_path: PathBuf, pub map: HashSet<(RustMap, MistMap)>, } ``` * `RustMap(usize, usize)` — line/column in generated Rust * `MistMap(usize, usize)` — line/column in original Mist ### Mapping Lifecycle [#mapping-lifecycle] 1. **Codegen** — Each `Spanned` AST node records its Mist position and the current Rust position in the codegen output buffer via `GenSpanTranslation` 2. **Persistence** — The mapping is serialized to `.map.json` alongside the transpiled `.rs` output 3. **Build-time remapping** — The builder reads `.map.json` to remap `cargo` diagnostics back to Mist positions 4. **LSP remapping** — Both Mist→Rust and Rust→Mist direction queries are supported via `find()` and `find_by_mist()` # Semantic Analysis (/docs/internals/semantic-analysis) The semantic checker (`semantics.rs`) performs class field initialization analysis. When a class has a constructor, it verifies that every declared field is mutated (directly or indirectly via method calls) within the constructor body. ### Field Initialization [#field-initialization] The primary semantic check ensures all class fields are initialized in the constructor: ```mist class Player { i32 health; str& name; pub constructor(str& name) { self.name = name; // Error: field 'health' is uninitialized } } ``` ### How It Works [#how-it-works] The `GetMutability` trait tracks which identifiers are mutated: 1. Collects all identifiers that get `&mut` references 2. Tracks `self.field = value` patterns 3. Follows method calls that might initialize fields (transitively) 4. Ensures all conditional branches initialize the same fields (intersection semantics) 5. `super` assignments count for `_super` field initialization If any field is uninitialized after the analysis, a `SemanticError` is reported with the field's source position. # Transpilation Pipeline (/docs/internals/transpiler) The transpiler (`transpiler.mist` in `mist_api`) orchestrates the full pipeline. ### Pipeline [#pipeline] 1. Read `Mist.toml` configuration 2. Build a `Module` tree from the filesystem (discovering `package.mist` files) 3. For each module: a. Parse Mist source b. Run semantic checks c. Generate Rust code and position mapping d. Write `.rs` file and `.map.json` file 4. Recursively transpile included crates ### Module Tree [#module-tree] The `Module` struct represents the file-to-module mapping: ```mist pub class Module { pub String name; pub PathBuf path; pub Vec children; pub constructor(PathBuf mist_path, MistConfig& config) { ... } pub bool is_package(&self) { ... } pub PathBuf output_dir(&self, PathBuf& parent_dir) { ... } pub PathBuf output_path(&self, PathBuf& parent_dir, ...) { ... } } ``` ### Transpilation Mapping Rules [#transpilation-mapping-rules] | Mist Source | Rust Output | | ---------------------- | ---------------------- | | `src/main.mist` | `.mist/src/main.rs` | | `src/foo.mist` | `.mist/src/foo.rs` | | `src/bar/package.mist` | `.mist/src/bar/mod.rs` | | `src/bar/baz.mist` | `.mist/src/bar/baz.rs` | A `package.mist` file generates `mod.rs` in the output directory and contains `pub mod ;` declarations for its children. ### Caching [#caching] The transpiler implements basic caching via file modification times: ```rust fn is_source_newer(source: &Path, output: &Path) -> io::Result { if !output.exists() { return Ok(true); } let source_time = fs::metadata(source)?.modified()?; let output_time = fs::metadata(output)?.modified()?; Ok(source_time > output_time) } ``` If a source file has not changed since the last transpilation, the `.rs` file is not regenerated.