### What's Changed
- Improve readability of unions and intersections by limiting the number
of elements of those types that can be presented on a single line (gated
under `FFlag::LuauToStringSimpleCompositeTypesSingleLine`)
- Adds a new option to the compiler `--record-stats` to record and
output compilation statistics
- `if...then...else` expressions are now optimized into `AND/OR` form
when possible.
### VM
- Add a new `buffer` type to Luau based on the [buffer
RFC](https://github.com/Roblox/luau/pull/739) and additional C API
functions to work with it; this release does not include the library.
- Internal C API to work with string buffers has been updated to align
with Lua version more closely
### Native Codegen
- Added support for new X64 instruction (rev) and new A64 instruction
(bswap) in the assembler
- Simplified the way numerical loop condition is translated to IR
### New Type Solver
- Operator inference now handled by type families
- Created a new system called `Type Paths` to explain why subtyping
tests fail in order to improve the quality of error messages.
- Systematic changes to implement Data Flow analysis in the new solver
(`Breadcrumb` removed and replaced with `RefinementKey`)
---
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
- Cleaned up `FFlag::FixFindBindingAtFunctionName`,
`FFlag::LuauNormalizeBlockedTypes`, `FFlag::LuauPCallDebuggerFix`
- Added support for break and continue into control flow analysis
- The old type unification engine will now report a more fine-grained
error at times, indicating that type normalization in particular failed
# New Type Solver
- Refactor of Unifier2, the new unification implementation for Luau
- Completed MVP of new unification implementation
- Dramatically simplified overload selection logic
- Type family reduction can now apply sooner to free types that have
been solved
- Subtyping now supports table indexers
- Generalization now replaces bad generics with unknown
# Native Code Generation
- Reduce stack spills caused by FINDUPVAL and STORE_TAG
- Improve Generate SHL/SHR/SAR/rotates with immediate operands in X64
- Removed redundant case re-check in table lookup fallback
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
* Fix a use-after-free bug in the new type cloning algorithm
* Tighten up the type of `coroutine.wrap`. It is now `<A..., R...>(f:
(A...) -> R...) -> ((A...) -> R...)`
* Break `.luaurc` out into a separate library target `Luau.Config`. This
makes it easier for applications to reason about config files without
also depending on the type inference engine.
* Move typechecking limits into `FrontendOptions`. This allows embedders
more finely-grained control over autocomplete's internal time limits.
* Fix stability issue with debugger onprotectederror callback allowing
break in non-yieldable contexts
New solver:
* Initial work toward [Local Type
Inference](0e1082108f/rfcs/local-type-inference.md)
* Introduce a new subtyping test. This will be much nicer than the old
test because it is completely separate both from actual type inference
and from error reporting.
Native code generation:
* Added function to compute iterated dominance frontier
* Optimize barriers in SET_UPVALUE when tag is known
* Cache lua_State::global in a register on A64
* Optimize constant stores in A64 lowering
* Track table array size state to optimize array size checks
* Add split tag/value store into a VM register
* Check that spills can outlive the block only in specific conditions
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Progress toward a diffing algorithm for types. We hope that this will
be useful for writing clearer error messages.
* Add a missing recursion limiter in `Unifier::tryUnifyTables`. This was
causing a crash in certain situations.
* Luau heap graph enumeration improvements:
* Weak references are not reported
* Added tag as a fallback name of non-string table links
* Included top Luau function information in thread name to understand
where thread might be suspended
* Constant folding for `math.pi` and `math.huge` at -O2
* Optimize `string.format` and `%*`
* This change makes string interpolation 1.5x-2x faster depending on the
number and type of formatted components, assuming a few are using
primitive types, and reduces associated GC pressure.
New type checker:
* Initial work toward tracking the upper and lower bounds of types
accurately.
Native code generation (JIT):
* Add IrCmd::CHECK_TRUTHY for improved assert fast-calls
* Do not compute type map for modules without types
* Capture metatable+readonly state for NEW_TABLE IR instructions
* Replace JUMP_CMP_ANY with CMP_ANY and existing JUMP_EQ_INT
* Add support for exits to VM with reentry lock in VmExit
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Type checker/autocomplete:
* `Luau::autocomplete` no longer performs typechecking internally, make
sure to run `Frontend::check` before performing autocomplete requests
* Autocomplete string suggestions without "" are now only suggested
inside the ""
* Autocomplete suggestions now include `function (anonymous autofilled)`
key with a full suggestion for the function expression (with arguments
included) stored in `AutocompleteEntry::insertText`
* `AutocompleteEntry::indexedWithSelf` is provided for function call
suggestions made with `:`
* Cyclic modules now see each other type exports as `any` to prevent
memory use-after-free (similar to module return type)
Runtime:
* Updated inline/loop unroll cost model to better handle assignments
(Fixes https://github.com/Roblox/luau/issues/978)
* `math.noise` speed was improved by ~30%
* `table.concat` speed was improved by ~5-7%
* `tonumber` and `tostring` now have fastcall paths that execute ~1.5x
and ~2.5x faster respectively (fixes#777)
* Fixed crash in `luaL_typename` when index refers to a non-existing
value
* Fixed potential out of memory scenario when using `string.sub` or
`string.char` in a loop
* Fixed behavior of some fastcall builtins when called without arguments
under -O2 to match original functions
* Support for native code execution in VM is now enabled by default
(note: native code still has to be generated explicitly)
* `Codegen::compile` now accepts `CodeGen_OnlyNativeModules` flag. When
set, only modules that have a `--!native` hot-comment at the top will be
compiled to native code
In our new typechecker:
* Generic type packs are no longer considered to be variadic during
unification
* Timeout and cancellation now works in new solver
* Fixed false positive errors around 'table' and 'function' type
refinements
* Table literals now use covariant unification rules. This is sound
since literal has no type specified and has no aliases
* Fixed issues with blocked types escaping the constraint solver
* Fixed more places where error messages that should've been suppressed
were still reported
* Fixed errors when iterating over a top table type
In our native code generation (jit):
* 'DebugLuauAbortingChecks' flag is now supported on A64
* LOP_NEWCLOSURE has been translated to IR
* Optimized operations like instantiation and module export for very
large types
In our new typechecker:
* Typechecking of function calls was rewritten to handle more cases
correctly
* Fixed a crash that can happen after self-referential type is exported
from a module
* Fixed a false positive error in string comparison
* Added handling of `for...in` variable type annotations and fixed
issues with the iterator call inside
* Self-referential 'hasProp' and 'setProp' constraints are now handled
correctly
In our native code generation (jit):
* Added '--target' argument to luau-compile to test multiple
architectures different from host architecture
* GC barrier tag check is skipped if type is already known to be
GC-collectable
* Added GET_TYPE/GET_TYPEOF instructions for type/typeof fast-calls
* Improved code size of interrupt handlers on X64
* Definition files can now ascribe indexers to class types.
(https://github.com/Roblox/luau/pull/949)
* Remove --compile support from the REPL. You can just use luau-compile
instead.
* When an exception is thrown during parallel typechecking (usually an
ICE), we now gracefully stop typechecking and drain active workers
before rethrowing the exception.
New solver
* Include more source location information when we hit an internal
compiler error
* Improve the logic that simplifies intersections of tables
JIT
* Save testable type annotations to bytecode
* Improve block placement for linearized blocks
* Add support for lea reg, [rip+offset] for labels
* Unify X64 and A64 codegen for RETURN
* Outline interrupt handlers for X64
* Remove global rArgN in favor of build.abi
* Change A64 INTERRUPT lowering to match X64
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed gcc warning about uninitialized `std::optional`
* Fixed inlining of functions when they are used to compute their own
arguments
In the new type solver:
* Type families that are not part of a function signature cannot be
resolved at instantiation time and will now produce an error. This will
be relaxed in the future when we get constraint clauses on function
signatures (internally)
* `never` type is now comparable
* Improved typechecking of `for..in` statements
* Fixed checks for number type in `Add` type family
* Performance was improved, with particularly large gains on large
projects
And in native code generation (jit):
* We eliminated the call instruction overhead when native code support
is enabled in the VM
* Small optimizations to arm64 lowering
* Reworked LOP_GETIMPORT handling to reduce assembly code size
* Fixed non-deterministic binary output
* Fixed bad code generation caused by incorrect SSA to VM register links
invalidation
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Lots of things going on this week:
* Fix a crash that could occur in the presence of a cyclic union. We
shouldn't be creating cyclic unions, but we shouldn't be crashing when
they arise either.
* Minor cleanup of `luau_precall`
* Internal change to make L->top handling slightly more uniform
* Optimize SETGLOBAL & GETGLOBAL fallback C functions.
* https://github.com/Roblox/luau/pull/929
* The syntax to the `luau-reduce` commandline tool has changed. It now
accepts a script, a command to execute, and an error to search for. It
no longer automatically passes the script to the command which makes it
a lot more flexible. Also be warned that it edits the script it is
passed **in place**. Do not point it at something that is not in source
control!
New solver
* Switch to a greedier but more fallible algorithm for simplifying union
and intersection types that are created as part of refinement
calculation. This has much better and more predictable performance.
* Fix a constraint cycle in recursive function calls.
* Much improved inference of binary addition. Functions like `function
add(x, y) return x + y end` can now be inferred without annotations. We
also accurately typecheck calls to functions like this.
* Many small bugfixes surrounding things like table indexers
* Add support for indexers on class types. This was previously added to
the old solver; we now add it to the new one for feature parity.
JIT
* https://github.com/Roblox/luau/pull/931
* Fuse key.value and key.tt loads for CEHCK_SLOT_MATCH in A64
* Implement remaining aliases of BFM for A64
* Implement new callinfo flag for A64
* Add instruction simplification for int->num->int conversion chains
* Don't even load execdata for X64 calls
* Treat opcode fallbacks the same as manually written fallbacks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* `ClassType` can now have an indexer defined on it. This allows custom
types to be used in `t[x]` expressions.
* Fixed search for closest executable breakpoint line. Previously,
breakpoints might have been skipped in `else` blocks at the end of a
function
* Fixed how unification is performed for two optional types `a? <: b?`,
previously it might have unified either 'a' or 'b' with 'nil'. Note that
this fix is not enabled by default yet (see the list in
`ExperimentalFlags.h`)
In the new type solver, a concept of 'Type Families' has been
introduced.
Type families can be thought of as type aliases with custom type
inference/reduction logic included with them.
For example, we can have an `Add<T, U>` type family that will resolve
the type that is the result of adding two values together.
This will help type inference to figure out what 'T' and 'U' might be
when explicit type annotations are not provided.
In this update we don't define any type families, but they will be added
in the near future.
It is also possible for Luau embedders to define their own type families
in the global/environment scope.
Other changes include:
* Fixed scope used to find out which generic types should be included in
the function generic type list
* Fixed a crash after cyclic bound types were created during unification
And in native code generation (jit):
* Use of arm64 target on M1 now requires macOS 13
* Entry into native code has been optimized. This is especially
important for coroutine call/pcall performance as they involve going
through a C call frame
* LOP_LOADK(X) translation into IR has been improved to enable type
tag/constant propagation
* arm64 can use integer immediate values to synthesize floating-point
values
* x64 assembler removes duplicate 64bit numbers from the data section to
save space
* Linux `perf` can now be used to profile native Luau code (when running
with --codegen-perf CLI argument)
* Work toward affording parallel type checking
* The interface to `LazyType` has changed:
* `LazyType` now takes a second callback that is passed the `LazyType&`
itself. This new callback is responsible for populating the field
`TypeId LazyType::unwrapped`. Multithreaded implementations should
acquire a lock in this callback.
* Modules now retain their `humanReadableNames`. This reduces the number
of cases where type checking has to call back to a `ModuleResolver`.
* https://github.com/Roblox/luau/pull/902
* Add timing info to the Luau REPL compilation output
We've also fixed some bugs and crashes in the new solver as we march
toward readiness.
* Thread ICEs (Internal Compiler Errors) back to the Frontend properly
* Refinements are no longer applied to lvalues
* More miscellaneous stability improvements
Lots of activity in the new JIT engine:
* Implement register spilling/restore for A64
* Correct Luau IR value restore location tracking
* Fixed use-after-free in x86 register allocator spill restore
* Use btz for bit tests
* Finish branch assembly support for A64
* Codesize and performance improvements for A64
* The bit32 library has been implemented for arm and x64
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Once again, all of our changes this week are for new type solver and the
JIT.
In the new type solver, we fixed cyclic type alias handling and multiple
stability issues.
In the JIT, our main progress was for arm64, where, after lowering 36%
of instructions, we start seeing first Luau functions executing
natively.
For x64, we performed code cleanup and refactoring to allow for future
optimizations.
* A small subset of control-flow refinements have been added to
recognize type options that are unreachable after a
conditional/unconditional code block. (Fixes
https://github.com/Roblox/luau/issues/356).
Some examples:
```lua
local function f(x: string?)
if not x then return end
-- x is 'string' here
end
```
Throwing calls like `error` or `assert(false)` instead of 'return' are
also recognized.
Existing complex refinements like type/typeof and tagged union checks
are expected to work, among others.
To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to
be removed from `ExperimentalFlags.h`.
If will become enabled unconditionally in the near future.
* Linter has been integrated into the typechecker analysis so that
type-aware lint warnings can work in any mode
`Frontend::lint` methods were deprecated, `Frontend::check` has to be
used instead with `runLintChecks` option set.
Resulting lint warning are located inside `CheckResult`.
* Fixed large performance drop and increased memory consumption when
array is filled at an offset (Fixes
https://github.com/Roblox/luau/issues/590)
* Part of [Type error suppression
RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md)
was implemented making subtyping checks with `any` type transitive.
---
In our work on the new type-solver:
* `--!nocheck` mode no longer reports type errors
* New solver will not be used for `--!nonstrict` modules until all
issues with strict mode typechecking are fixed
* Added control-flow aware type refinements mentioned earlier
In native code generation:
* `LOP_NAMECALL` has been translated to IR
* `type` and `typeof` builtin fastcalls have been translated to
IR/assembly
* Additional steps were taken towards arm64 support
* Fixed incorrect lexeme generated for string parts in the middle of an
interpolated string (Fixes https://github.com/Roblox/luau/issues/744)
* DeprecatedApi lint can report some issues without type inference
information
* Fixed performance of autocomplete requests when suggestions have large
intersection types (Solves
https://github.com/Roblox/luau/discussions/847)
* Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC:
Deprecate
table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md))
* With -O2 optimization level, we now optimize builtin calls based on
known argument/return count.
Note that this change can be observable if `getfenv/setfenv` is used to
substitute a builtin, especially if arity is different.
Fastcall heavy tests show a 1-2% improvement.
* Luau can now be built with clang-cl (Fixes
https://github.com/Roblox/luau/issues/736)
We also made many improvements to our experimental components.
For our new type solver:
* Overhauled data flow analysis system, fixed issues with 'repeat'
loops, global variables and type annotations
* Type refinements now work on generic table indexing with a string
literal
* Type refinements will properly track potentially 'nil' values (like
t[x] for a missing key) and their further refinements
* Internal top table type is now isomorphic to `{}` which fixes issues
when `typeof(v) == 'table'` type refinement is handled
* References to non-existent types in type annotations no longer resolve
to 'error' type like in old solver
* Improved handling of class unions in property access expressions
* Fixed default type packs
* Unsealed tables can now have metatables
* Restored expected types for function arguments
And for native code generation:
* Added min and max IR instructions mapping to vminsd/vmaxsd on x64
* We now speculatively extract direct execution fast-paths based on
expected types of expressions which provides better optimization
opportunities inside a single basic block
* Translated existing math fastcalls to IR form to improve tag guard
removal and constant propagation
We've made a few small changes to reduce the amount of stack we use when
typechecking nested method calls (eg `foo:bar():baz():quux()`).
We've also fixed a small bytecode compiler issue that caused us to emit
redundant jump instructions in code that conditionally uses `break` or
`continue`.
On the new solver, we've switched to a new, better way to handle
augmentations to unsealed tables. We've also made some substantial
improvements to type inference and error reporting on function calls.
These things should both be on par with the old solver now.
The main improvements to the native code generator have been elimination
of some redundant type tag checks. Also, we are starting to inline
particular fastcalls directly to IR.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
This week we only have updates to new type solver and JIT. Both projects
are still in the process of being built out. Neither are ready for
general use yet.
In the new solver, we fixed issues with recursive type aliases.
Duplicated type parameters are once again reported, exported types are
being recorder and function argument names are placed inside function
types.
We also made improvements to restore parts of bidirectional type
tracking.
On native code generation side, namecall instruction lowering was fixed,
we fixed inconsistencies in IR command definitions and added utility
function to help with constant folding.
* For autocomplete, additional information is included in Scope for type
alias name locations and names of imported modules
* Improved autocomplete suggestions in 'for' and 'while' loop headers
* String match functions return types are now optional strings and
numbers because match is not guaranteed at runtime
* Fixed build issue on gcc 11 and up (Fixes
https://github.com/Roblox/luau/issues/806)
* Fixed unions of `nil` types displaying as `?`
* Internal normalization now handles class types which can make
previously failing (incorrectly) sub-typing checks to succeed
* The AST JSON encoder will now stringify infinity and NaN according to the JSON5 spec using the tokens `Infinity`, `-Infinity`, and `NaN`.
* Improve autocompletion of table keys if the type of that key is a union of string singletons.
* Type inference of `a and b` and `a or b` has been improved (Fixes
https://github.com/Roblox/luau/issues/730)
* Improved error message when `for ... in x` loop iterates over a value
that could be 'nil'
* Return type of `next` not includes 'nil' (Fixes
https://github.com/Roblox/luau/issues/706)
* Improved type inference of `string` type
* Luau library table type names are now reported as `typeof(string)`/etc
instead of just `string` which was misleading
* Added parsing error when optional type alias type parameter wasn't
provided after `=` token
* Improved tagged union type refinement in conditional expressions, type
in `else` branch should no longer include previously handled union
options
* Reduce the stack utilization of type checking.
* Improve the error message that's reported when a delimiting comma is
missing from a table literal. eg
```lua
local t = {
first = 1
second = 2
}```
* Support `["prop"]` syntax on class definitions in definition files.
(#704)
* Improve type checking performance for complex overloaded functions
* Fix rare cases of incorrect stack traces for out of memory errors at
runtime
* Reoptimized math.min/max/bit32 builtins assuming at least 2 arguments are used (1-2% lift on some benchmarks)
* Type errors that mention function types no longer have redundant parenthesis around return type
* Luau REPL now supports --compile=remarks which displays the source code with optimization remarks embedded as comments
* Builtin calls are slightly faster when called with 1-2 arguments (~1% improvement in some benchmarks)
- Fix rare type checking bugs with invalid generic types escaping the
module scope
- Fix type checking of variadic type packs in certain cases
- Implement type normalization, which resolves a large set of various
issues with unions/intersections in type checker
- Improve parse errors for trailing commas in function calls and type
lists
- Reduce profiling skew when using --profile with very high frequencies
- Improve performance of `lua_getinfo` (`debug.info`, `debug.traceback`
and profiling overhead are now 20% faster/smaller)
- Improve performance of polymorphic comparisons (1-2% lift on some
benchmarks)
- Improve performance of closure creation (1-2% lift on some benchmarks)
- Improve string comparison performance (4% lift on string sorting)
- Type aliases can no longer override primitive types; attempts to do
that will result in a type error
- Fix misleading type error messages for mismatches in expression list
length during assignment
- Fix incorrect type name display in certain cases
- setmetatable/getmetatable are now ~2x faster
- tools/perfstat.py can be used to display statistics about profiles
captured via --profile switch
- Improve type error messages for argument count mismatch in certain
cases
- Fix type checking variadic returns when the type is incompatible which
type checked in certain cases
- Reduce size of upvalue objects by 8 bytes on 64-bit platforms
- Reduce I$ footprint of interpreter by 1.5KB
- Reduce GC pause during atomic stage for programs with a lot of threads
- Remove support for bytecode v2
- Remove type definitions of
`utf8.nfcnormalize`/`nfdnormalize`/`graphemes` that aren't supported by
standalone Luau library
- Add `lua_costatus` to retrieve extended thread status (similar to
`coroutine.status`)
- Improve GC sweeping performance (2-10% improvement on allocation-heavy
benchmarks)
- Improve ComparisonPrecedence lint suggestions for three-way comparisons (X < Y < Z)
- Improve type checking stability
- Improve location information for errors when parsing invalid type annotations
- Compiler now generates bytecode version 3 in all configurations
- Improve performance of comparisons against numeric constants on AArch64
- Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative
- Fix an off-by-one error in type error text for incorrect use of string.format
- Reduce stack consumption further during parsing, hopefully eliminating stack overflows during parsing/compilation for good
- Mark interpolated string support as experimental (requires --fflags=LuauInterpolatedStringBaseSupport to enable)
- Simplify garbage collection treatment of upvalues, reducing cache misses during sweeping stage and reducing the cost of upvalue assignment (SETUPVAL); supersedes #643
- Simplify garbage collection treatment of sleeping threads
- Simplify sweeping of alive threads, reducing cache misses during sweeping stage
- Simplify management of string buffers, removing redundant linked list operations
- Fix autocomplete not suggesting globals defined after the cursor (fixes#622)
- Improve type checker stability
- Reduce parser C stack consumption which fixes some stack overflow crashes on deeply nested sources
- Improve performance of bit32.extract/replace when width is implied (~3% faster chess)
- Improve performance of bit32.extract when field/width are constants (~10% faster base64)
- Heap dump now annotates thread stacks with local variable/function names
* Run clang-format
* Contains a preliminary implementation of deferred constraint resolution
* Reduce stack usage by some recursive functions
* Fix a bug when smartCloning a BoundTypeVar
* Remove some GC related flags from VM
