luau/tests/Normalize.test.cpp

987 lines
27 KiB
C++
Raw Normal View History

2022-04-15 07:57:43 +08:00
// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details
#include "Fixture.h"
Sync to upstream/release/562 (#828) * Fixed rare use-after-free in analysis during table unification A lot of work these past months went into two new Luau components: * A near full rewrite of the typechecker using a new deferred constraint resolution system * Native code generation for AoT/JiT compilation of VM bytecode into x64 (avx)/arm64 instructions Both of these components are far from finished and we don't provide documentation on building and using them at this point. However, curious community members expressed interest in learning about changes that go into these components each week, so we are now listing them here in the 'sync' pull request descriptions. --- New typechecker can be enabled by setting DebugLuauDeferredConstraintResolution flag to 'true'. It is considered unstable right now, so try it at your own risk. Even though it already provides better type inference than the current one in some cases, our main goal right now is to reach feature parity with current typechecker. Features which improve over the capabilities of the current typechecker are marked as '(NEW)'. Changes to new typechecker: * Regular for loop index and parameters are now typechecked * Invalid type annotations on local variables are ignored to improve autocomplete * Fixed missing autocomplete type suggestions for function arguments * Type reduction is now performed to produce simpler types to be presented to the user (error messages, custom LSPs) * Internally, complex types like '((number | string) & ~(false?)) | string' can be produced, which is just 'string | number' when simplified * Fixed spots where support for unknown and never types was missing * (NEW) Length operator '#' is now valid to use on top table type, this type comes up when doing typeof(x) == "table" guards and isn't available in current typechecker --- Changes to native code generation: * Additional math library fast calls are now lowered to x64: math.ldexp, math.round, math.frexp, math.modf, math.sign and math.clamp
2023-02-04 03:26:13 +08:00
#include "Luau/AstQuery.h"
2022-09-24 03:17:25 +08:00
#include "Luau/Common.h"
#include "Luau/Type.h"
2022-04-15 07:57:43 +08:00
#include "doctest.h"
#include "Luau/Normalize.h"
#include "Luau/BuiltinDefinitions.h"
Sync to upstream/release/562 (#828) * Fixed rare use-after-free in analysis during table unification A lot of work these past months went into two new Luau components: * A near full rewrite of the typechecker using a new deferred constraint resolution system * Native code generation for AoT/JiT compilation of VM bytecode into x64 (avx)/arm64 instructions Both of these components are far from finished and we don't provide documentation on building and using them at this point. However, curious community members expressed interest in learning about changes that go into these components each week, so we are now listing them here in the 'sync' pull request descriptions. --- New typechecker can be enabled by setting DebugLuauDeferredConstraintResolution flag to 'true'. It is considered unstable right now, so try it at your own risk. Even though it already provides better type inference than the current one in some cases, our main goal right now is to reach feature parity with current typechecker. Features which improve over the capabilities of the current typechecker are marked as '(NEW)'. Changes to new typechecker: * Regular for loop index and parameters are now typechecked * Invalid type annotations on local variables are ignored to improve autocomplete * Fixed missing autocomplete type suggestions for function arguments * Type reduction is now performed to produce simpler types to be presented to the user (error messages, custom LSPs) * Internally, complex types like '((number | string) & ~(false?)) | string' can be produced, which is just 'string | number' when simplified * Fixed spots where support for unknown and never types was missing * (NEW) Length operator '#' is now valid to use on top table type, this type comes up when doing typeof(x) == "table" guards and isn't available in current typechecker --- Changes to native code generation: * Additional math library fast calls are now lowered to x64: math.ldexp, math.round, math.frexp, math.modf, math.sign and math.clamp
2023-02-04 03:26:13 +08:00
LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution)
Sync to upstream/release/622 (#1232) # What's changed? * Improved the actual message for the type errors for `cannot call non-function` when attempting to call a union of functions/callable tables. The error now correctly explains the issue is an inability to determine the return type of the call in this situation. * Resolve an issue where tables and metatables were not correctly being cloned during instantiation (fixes #1176). * Refactor `luaM_getnextgcopage` to `luaM_getnextpage` (generally removing `gco` prefix where appropriate). * Optimize `table.move` between tables for large ranges of elements. * Reformat a bunch of code automatically using `clang-format`. ### New Type Solver * Clean up minimally-used or unused constraints in the constraint solver (`InstantiationConstraint`, `SetOpConstraint`, `SingletonOrTopTypeConstraint`). * Add a builtin `singleton` type family to replace `SingletonOrTopTypeConstraint` when inferring refinements. * Fixed a crash involving type path reasoning by recording when type family reduction has taken place in the path. * Improved constraint ordering by blocking on unreduced types families that are not yet proven uninhabitable. * Improved the handling of `SetIndexerConstraints` for both better inference quality and to resolve crashes. * Fix a crash when normalizing cyclic unions of intersections. * Fix a crash when normalizing an intersection with the negation of `unknown`. * Fix a number of crashes caused by missing `follow` calls on `TypeId`s. * Changed type family reduction to correctly use a semantic notion of uninhabited types, rather than checking for `never` types specifically. * Refactor the `union` and `intersect` type families to be variadic. ### Native Code Generation * Improve translation for userdata key get/set and userdata/vector namecall. * Provide `[top level]` and `[anonymous]` as function names to `FunctionStats` as appropriate when no function name is available. * Disable unwind support on Android platforms since it is unsupported. * --- ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-04-20 05:48:02 +08:00
LUAU_FASTFLAG(LuauNormalizeNotUnknownIntersection)
LUAU_FASTINT(LuauTypeInferRecursionLimit)
2022-04-15 07:57:43 +08:00
using namespace Luau;
namespace
{
struct IsSubtypeFixture : Fixture
2022-04-15 07:57:43 +08:00
{
bool isSubtype(TypeId a, TypeId b)
{
ModulePtr module = getMainModule();
REQUIRE(module);
if (!module->hasModuleScope())
FAIL("isSubtype: module scope data is not available");
return ::Luau::isSubtype(a, b, NotNull{module->getModuleScope().get()}, builtinTypes, ice);
}
2022-04-15 07:57:43 +08:00
};
} // namespace
2022-04-15 07:57:43 +08:00
TEST_SUITE_BEGIN("isSubtype");
TEST_CASE_FIXTURE(IsSubtypeFixture, "primitives")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a = 41
local b = 32
local c = "hello"
local d = "world"
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
TypeId d = requireType("d");
CHECK(isSubtype(b, a));
CHECK(isSubtype(d, c));
CHECK(!isSubtype(d, a));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "functions")
2022-04-15 07:57:43 +08:00
{
check(R"(
function a(x: number): number return x end
function b(x: number): number return x end
function c(x: number?): number return x end
function d(x: number): number? return x end
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
TypeId d = requireType("d");
CHECK(isSubtype(b, a));
CHECK(isSubtype(c, a));
CHECK(!isSubtype(d, a));
CHECK(isSubtype(a, d));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "variadic_functions_with_no_head")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: (...number) -> ()
local b: (...number?) -> ()
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
CHECK(isSubtype(b, a));
CHECK(!isSubtype(a, b));
}
#if 0
TEST_CASE_FIXTURE(IsSubtypeFixture, "variadic_function_with_head")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: (...number) -> ()
local b: (number, number) -> ()
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
CHECK(!isSubtype(b, a));
CHECK(isSubtype(a, b));
}
#endif
TEST_CASE_FIXTURE(IsSubtypeFixture, "union")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: number | string
local b: number
local c: string
local d: number?
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
TypeId d = requireType("d");
CHECK(isSubtype(b, a));
CHECK(!isSubtype(a, b));
CHECK(isSubtype(c, a));
CHECK(!isSubtype(a, c));
CHECK(!isSubtype(d, a));
CHECK(!isSubtype(a, d));
CHECK(isSubtype(b, d));
CHECK(!isSubtype(d, b));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "table_with_union_prop")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: {x: number}
local b: {x: number?}
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
if (FFlag::DebugLuauDeferredConstraintResolution)
CHECK(!isSubtype(a, b)); // table properties are invariant
else
CHECK(isSubtype(a, b));
2022-04-15 07:57:43 +08:00
CHECK(!isSubtype(b, a));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "table_with_any_prop")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: {x: number}
local b: {x: any}
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
if (FFlag::DebugLuauDeferredConstraintResolution)
CHECK(!isSubtype(a, b)); // table properties are invariant
else
CHECK(isSubtype(a, b));
Sync to upstream/release/568 (#865) * A small subset of control-flow refinements have been added to recognize type options that are unreachable after a conditional/unconditional code block. (Fixes https://github.com/Roblox/luau/issues/356). Some examples: ```lua local function f(x: string?) if not x then return end -- x is 'string' here end ``` Throwing calls like `error` or `assert(false)` instead of 'return' are also recognized. Existing complex refinements like type/typeof and tagged union checks are expected to work, among others. To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to be removed from `ExperimentalFlags.h`. If will become enabled unconditionally in the near future. * Linter has been integrated into the typechecker analysis so that type-aware lint warnings can work in any mode `Frontend::lint` methods were deprecated, `Frontend::check` has to be used instead with `runLintChecks` option set. Resulting lint warning are located inside `CheckResult`. * Fixed large performance drop and increased memory consumption when array is filled at an offset (Fixes https://github.com/Roblox/luau/issues/590) * Part of [Type error suppression RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md) was implemented making subtyping checks with `any` type transitive. --- In our work on the new type-solver: * `--!nocheck` mode no longer reports type errors * New solver will not be used for `--!nonstrict` modules until all issues with strict mode typechecking are fixed * Added control-flow aware type refinements mentioned earlier In native code generation: * `LOP_NAMECALL` has been translated to IR * `type` and `typeof` builtin fastcalls have been translated to IR/assembly * Additional steps were taken towards arm64 support
2023-03-18 03:20:37 +08:00
CHECK(!isSubtype(b, a));
2022-04-15 07:57:43 +08:00
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "intersection")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: number & string
local b: number
local c: string
local d: number & nil
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
TypeId d = requireType("d");
CHECK(!isSubtype(b, a));
CHECK(isSubtype(a, b));
CHECK(!isSubtype(c, a));
CHECK(isSubtype(a, c));
// These types are both equivalent to never
CHECK(isSubtype(d, a));
CHECK(isSubtype(a, d));
2022-04-15 07:57:43 +08:00
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "union_and_intersection")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: number & string
local b: number | nil
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
CHECK(!isSubtype(b, a));
CHECK(isSubtype(a, b));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "tables")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: {x: number}
local b: {x: any}
local c: {y: number}
local d: {x: number, y: number}
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
TypeId d = requireType("d");
if (FFlag::DebugLuauDeferredConstraintResolution)
CHECK(!isSubtype(a, b)); // table properties are invariant
else
CHECK(isSubtype(a, b));
Sync to upstream/release/568 (#865) * A small subset of control-flow refinements have been added to recognize type options that are unreachable after a conditional/unconditional code block. (Fixes https://github.com/Roblox/luau/issues/356). Some examples: ```lua local function f(x: string?) if not x then return end -- x is 'string' here end ``` Throwing calls like `error` or `assert(false)` instead of 'return' are also recognized. Existing complex refinements like type/typeof and tagged union checks are expected to work, among others. To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to be removed from `ExperimentalFlags.h`. If will become enabled unconditionally in the near future. * Linter has been integrated into the typechecker analysis so that type-aware lint warnings can work in any mode `Frontend::lint` methods were deprecated, `Frontend::check` has to be used instead with `runLintChecks` option set. Resulting lint warning are located inside `CheckResult`. * Fixed large performance drop and increased memory consumption when array is filled at an offset (Fixes https://github.com/Roblox/luau/issues/590) * Part of [Type error suppression RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md) was implemented making subtyping checks with `any` type transitive. --- In our work on the new type-solver: * `--!nocheck` mode no longer reports type errors * New solver will not be used for `--!nonstrict` modules until all issues with strict mode typechecking are fixed * Added control-flow aware type refinements mentioned earlier In native code generation: * `LOP_NAMECALL` has been translated to IR * `type` and `typeof` builtin fastcalls have been translated to IR/assembly * Additional steps were taken towards arm64 support
2023-03-18 03:20:37 +08:00
CHECK(!isSubtype(b, a));
2022-04-15 07:57:43 +08:00
CHECK(!isSubtype(c, a));
CHECK(!isSubtype(a, c));
CHECK(isSubtype(d, a));
CHECK(!isSubtype(a, d));
if (FFlag::DebugLuauDeferredConstraintResolution)
CHECK(!isSubtype(d, b)); // table properties are invariant
else
CHECK(isSubtype(d, b));
2022-04-15 07:57:43 +08:00
CHECK(!isSubtype(b, d));
}
2022-04-22 05:44:27 +08:00
#if 0
TEST_CASE_FIXTURE(IsSubtypeFixture, "table_indexers_are_invariant")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: {[string]: number}
local b: {[string]: any}
local c: {[string]: number}
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
CHECK(!isSubtype(b, a));
CHECK(!isSubtype(a, b));
CHECK(isSubtype(c, a));
CHECK(isSubtype(a, c));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "mismatched_indexers")
2022-04-15 07:57:43 +08:00
{
check(R"(
local a: {x: number}
local b: {[string]: number}
local c: {}
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
CHECK(isSubtype(b, a));
CHECK(!isSubtype(a, b));
CHECK(!isSubtype(c, b));
CHECK(isSubtype(b, c));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "cyclic_table")
2022-04-15 07:57:43 +08:00
{
check(R"(
type A = {method: (A) -> ()}
local a: A
type B = {method: (any) -> ()}
local b: B
type C = {method: (C) -> ()}
local c: C
type D = {method: (D) -> (), another: (D) -> ()}
local d: D
type E = {method: (A) -> (), another: (E) -> ()}
local e: E
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
TypeId d = requireType("d");
TypeId e = requireType("e");
CHECK(isSubtype(b, a));
CHECK(!isSubtype(a, b));
CHECK(isSubtype(c, a));
CHECK(isSubtype(a, c));
CHECK(!isSubtype(d, a));
CHECK(!isSubtype(a, d));
CHECK(isSubtype(e, a));
CHECK(!isSubtype(a, e));
}
#endif
TEST_CASE_FIXTURE(IsSubtypeFixture, "classes")
2022-04-15 07:57:43 +08:00
{
createSomeClasses(&frontend);
2022-04-15 07:57:43 +08:00
check(""); // Ensure that we have a main Module.
TypeId p = frontend.globals.globalScope->lookupType("Parent")->type;
TypeId c = frontend.globals.globalScope->lookupType("Child")->type;
TypeId u = frontend.globals.globalScope->lookupType("Unrelated")->type;
2022-04-15 07:57:43 +08:00
CHECK(isSubtype(c, p));
CHECK(!isSubtype(p, c));
CHECK(!isSubtype(u, p));
CHECK(!isSubtype(p, u));
}
#if 0
TEST_CASE_FIXTURE(IsSubtypeFixture, "metatable" * doctest::expected_failures{1})
2022-04-15 07:57:43 +08:00
{
check(R"(
local T = {}
T.__index = T
function T.new()
return setmetatable({}, T)
end
function T:method() end
local a: typeof(T.new)
local b: {method: (any) -> ()}
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
CHECK(isSubtype(a, b));
}
#endif
Sync to upstream/release/568 (#865) * A small subset of control-flow refinements have been added to recognize type options that are unreachable after a conditional/unconditional code block. (Fixes https://github.com/Roblox/luau/issues/356). Some examples: ```lua local function f(x: string?) if not x then return end -- x is 'string' here end ``` Throwing calls like `error` or `assert(false)` instead of 'return' are also recognized. Existing complex refinements like type/typeof and tagged union checks are expected to work, among others. To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to be removed from `ExperimentalFlags.h`. If will become enabled unconditionally in the near future. * Linter has been integrated into the typechecker analysis so that type-aware lint warnings can work in any mode `Frontend::lint` methods were deprecated, `Frontend::check` has to be used instead with `runLintChecks` option set. Resulting lint warning are located inside `CheckResult`. * Fixed large performance drop and increased memory consumption when array is filled at an offset (Fixes https://github.com/Roblox/luau/issues/590) * Part of [Type error suppression RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md) was implemented making subtyping checks with `any` type transitive. --- In our work on the new type-solver: * `--!nocheck` mode no longer reports type errors * New solver will not be used for `--!nonstrict` modules until all issues with strict mode typechecking are fixed * Added control-flow aware type refinements mentioned earlier In native code generation: * `LOP_NAMECALL` has been translated to IR * `type` and `typeof` builtin fastcalls have been translated to IR/assembly * Additional steps were taken towards arm64 support
2023-03-18 03:20:37 +08:00
TEST_CASE_FIXTURE(IsSubtypeFixture, "any_is_unknown_union_error")
{
check(R"(
local err = 5.nope.nope -- err is now an error type
local a : any
local b : (unknown | typeof(err))
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
CHECK(isSubtype(a, b));
CHECK(isSubtype(b, a));
CHECK_EQ("*error-type*", toString(requireType("err")));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "any_intersect_T_is_T")
{
check(R"(
local a : (any & string)
local b : string
local c : number
)");
TypeId a = requireType("a");
TypeId b = requireType("b");
TypeId c = requireType("c");
CHECK(isSubtype(a, b));
CHECK(isSubtype(b, a));
CHECK(!isSubtype(a, c));
CHECK(!isSubtype(c, a));
}
TEST_CASE_FIXTURE(IsSubtypeFixture, "error_suppression")
{
check("");
TypeId any = builtinTypes->anyType;
TypeId err = builtinTypes->errorType;
TypeId str = builtinTypes->stringType;
TypeId unk = builtinTypes->unknownType;
CHECK(!isSubtype(any, err));
CHECK(isSubtype(err, any));
CHECK(!isSubtype(any, str));
CHECK(isSubtype(str, any));
CHECK(!isSubtype(any, unk));
CHECK(isSubtype(unk, any));
CHECK(!isSubtype(err, str));
CHECK(!isSubtype(str, err));
CHECK(!isSubtype(err, unk));
CHECK(!isSubtype(unk, err));
CHECK(isSubtype(str, unk));
CHECK(!isSubtype(unk, str));
}
2022-04-15 07:57:43 +08:00
TEST_SUITE_END();
struct NormalizeFixture : Fixture
{
TypeArena arena;
InternalErrorReporter iceHandler;
UnifierSharedState unifierState{&iceHandler};
Normalizer normalizer{&arena, builtinTypes, NotNull{&unifierState}};
Scope globalScope{builtinTypes->anyTypePack};
NormalizeFixture()
{
registerHiddenTypes(&frontend);
}
std::shared_ptr<const NormalizedType> toNormalizedType(const std::string& annotation)
{
normalizer.clearCaches();
CheckResult result = check("type _Res = " + annotation);
LUAU_REQUIRE_NO_ERRORS(result);
Sync to upstream/release/562 (#828) * Fixed rare use-after-free in analysis during table unification A lot of work these past months went into two new Luau components: * A near full rewrite of the typechecker using a new deferred constraint resolution system * Native code generation for AoT/JiT compilation of VM bytecode into x64 (avx)/arm64 instructions Both of these components are far from finished and we don't provide documentation on building and using them at this point. However, curious community members expressed interest in learning about changes that go into these components each week, so we are now listing them here in the 'sync' pull request descriptions. --- New typechecker can be enabled by setting DebugLuauDeferredConstraintResolution flag to 'true'. It is considered unstable right now, so try it at your own risk. Even though it already provides better type inference than the current one in some cases, our main goal right now is to reach feature parity with current typechecker. Features which improve over the capabilities of the current typechecker are marked as '(NEW)'. Changes to new typechecker: * Regular for loop index and parameters are now typechecked * Invalid type annotations on local variables are ignored to improve autocomplete * Fixed missing autocomplete type suggestions for function arguments * Type reduction is now performed to produce simpler types to be presented to the user (error messages, custom LSPs) * Internally, complex types like '((number | string) & ~(false?)) | string' can be produced, which is just 'string | number' when simplified * Fixed spots where support for unknown and never types was missing * (NEW) Length operator '#' is now valid to use on top table type, this type comes up when doing typeof(x) == "table" guards and isn't available in current typechecker --- Changes to native code generation: * Additional math library fast calls are now lowered to x64: math.ldexp, math.round, math.frexp, math.modf, math.sign and math.clamp
2023-02-04 03:26:13 +08:00
if (FFlag::DebugLuauDeferredConstraintResolution)
{
SourceModule* sourceModule = getMainSourceModule();
REQUIRE(sourceModule);
AstNode* node = findNodeAtPosition(*sourceModule, {0, 5});
REQUIRE(node);
AstStatTypeAlias* alias = node->as<AstStatTypeAlias>();
REQUIRE(alias);
Sync to upstream/release/577 (#934) Lots of things going on this week: * Fix a crash that could occur in the presence of a cyclic union. We shouldn't be creating cyclic unions, but we shouldn't be crashing when they arise either. * Minor cleanup of `luau_precall` * Internal change to make L->top handling slightly more uniform * Optimize SETGLOBAL & GETGLOBAL fallback C functions. * https://github.com/Roblox/luau/pull/929 * The syntax to the `luau-reduce` commandline tool has changed. It now accepts a script, a command to execute, and an error to search for. It no longer automatically passes the script to the command which makes it a lot more flexible. Also be warned that it edits the script it is passed **in place**. Do not point it at something that is not in source control! New solver * Switch to a greedier but more fallible algorithm for simplifying union and intersection types that are created as part of refinement calculation. This has much better and more predictable performance. * Fix a constraint cycle in recursive function calls. * Much improved inference of binary addition. Functions like `function add(x, y) return x + y end` can now be inferred without annotations. We also accurately typecheck calls to functions like this. * Many small bugfixes surrounding things like table indexers * Add support for indexers on class types. This was previously added to the old solver; we now add it to the new one for feature parity. JIT * https://github.com/Roblox/luau/pull/931 * Fuse key.value and key.tt loads for CEHCK_SLOT_MATCH in A64 * Implement remaining aliases of BFM for A64 * Implement new callinfo flag for A64 * Add instruction simplification for int->num->int conversion chains * Don't even load execdata for X64 calls * Treat opcode fallbacks the same as manually written fallbacks --------- Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-05-20 03:37:30 +08:00
TypeId* originalTy = getMainModule()->astResolvedTypes.find(alias->type);
Sync to upstream/release/562 (#828) * Fixed rare use-after-free in analysis during table unification A lot of work these past months went into two new Luau components: * A near full rewrite of the typechecker using a new deferred constraint resolution system * Native code generation for AoT/JiT compilation of VM bytecode into x64 (avx)/arm64 instructions Both of these components are far from finished and we don't provide documentation on building and using them at this point. However, curious community members expressed interest in learning about changes that go into these components each week, so we are now listing them here in the 'sync' pull request descriptions. --- New typechecker can be enabled by setting DebugLuauDeferredConstraintResolution flag to 'true'. It is considered unstable right now, so try it at your own risk. Even though it already provides better type inference than the current one in some cases, our main goal right now is to reach feature parity with current typechecker. Features which improve over the capabilities of the current typechecker are marked as '(NEW)'. Changes to new typechecker: * Regular for loop index and parameters are now typechecked * Invalid type annotations on local variables are ignored to improve autocomplete * Fixed missing autocomplete type suggestions for function arguments * Type reduction is now performed to produce simpler types to be presented to the user (error messages, custom LSPs) * Internally, complex types like '((number | string) & ~(false?)) | string' can be produced, which is just 'string | number' when simplified * Fixed spots where support for unknown and never types was missing * (NEW) Length operator '#' is now valid to use on top table type, this type comes up when doing typeof(x) == "table" guards and isn't available in current typechecker --- Changes to native code generation: * Additional math library fast calls are now lowered to x64: math.ldexp, math.round, math.frexp, math.modf, math.sign and math.clamp
2023-02-04 03:26:13 +08:00
REQUIRE(originalTy);
return normalizer.normalize(*originalTy);
}
else
{
std::optional<TypeId> ty = lookupType("_Res");
REQUIRE(ty);
return normalizer.normalize(*ty);
}
}
TypeId normal(const std::string& annotation)
{
std::shared_ptr<const NormalizedType> norm = toNormalizedType(annotation);
REQUIRE(norm);
return normalizer.typeFromNormal(*norm);
}
};
2022-04-15 07:57:43 +08:00
TEST_SUITE_BEGIN("Normalize");
TEST_CASE_FIXTURE(NormalizeFixture, "string_intersection_is_commutative")
{
auto c4 = toString(normal(R"(
string & (string & Not<"a"> & Not<"b">)
)"));
auto c4Reverse = toString(normal(R"(
(string & Not<"a"> & Not<"b">) & string
)"));
CHECK(c4 == c4Reverse);
CHECK_EQ("string & ~\"a\" & ~\"b\"", c4);
auto c5 = toString(normal(R"(
(string & Not<"a"> & Not<"b">) & (string & Not<"b"> & Not<"c">)
)"));
auto c5Reverse = toString(normal(R"(
(string & Not<"b"> & Not<"c">) & (string & Not<"a"> & Not<"c">)
)"));
CHECK(c5 == c5Reverse);
CHECK_EQ("string & ~\"a\" & ~\"b\" & ~\"c\"", c5);
auto c6 = toString(normal(R"(
("a" | "b") & (string & Not<"b"> & Not<"c">)
)"));
auto c6Reverse = toString(normal(R"(
(string & Not<"b"> & Not<"c">) & ("a" | "b")
)"));
CHECK(c6 == c6Reverse);
CHECK_EQ("\"a\"", c6);
auto c7 = toString(normal(R"(
string & ("b" | "c")
)"));
auto c7Reverse = toString(normal(R"(
("b" | "c") & string
)"));
CHECK(c7 == c7Reverse);
CHECK_EQ("\"b\" | \"c\"", c7);
auto c8 = toString(normal(R"(
(string & Not<"a"> & Not<"b">) & ("b" | "c")
)"));
auto c8Reverse = toString(normal(R"(
("b" | "c") & (string & Not<"a"> & Not<"b">)
)"));
CHECK(c8 == c8Reverse);
CHECK_EQ("\"c\"", c8);
auto c9 = toString(normal(R"(
("a" | "b") & ("b" | "c")
)"));
auto c9Reverse = toString(normal(R"(
("b" | "c") & ("a" | "b")
)"));
CHECK(c9 == c9Reverse);
CHECK_EQ("\"b\"", c9);
auto l = toString(normal(R"(
(string | number) & ("a" | true)
)"));
auto r = toString(normal(R"(
("a" | true) & (string | number)
)"));
CHECK(l == r);
CHECK_EQ("\"a\"", l);
}
TEST_CASE_FIXTURE(NormalizeFixture, "negate_string")
{
CHECK("number" == toString(normal(R"(
(number | string) & Not<string>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negate_string_from_cofinite_string_intersection")
{
CHECK("number" == toString(normal(R"(
(number | (string & Not<"hello"> & Not<"world">)) & Not<string>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "no_op_negation_is_dropped")
{
CHECK("number" == toString(normal(R"(
number & Not<string>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "union_of_negation")
{
CHECK("string" == toString(normal(R"(
(string & Not<"hello">) | "hello"
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "intersect_truthy")
{
CHECK("number | string | true" == toString(normal(R"(
(string | number | boolean | nil) & Not<false | nil>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "intersect_truthy_expressed_as_intersection")
{
CHECK("number | string | true" == toString(normal(R"(
(string | number | boolean | nil) & Not<false> & Not<nil>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "union_of_union")
{
CHECK(R"("alpha" | "beta" | "gamma")" == toString(normal(R"(
("alpha" | "beta") | "gamma"
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "union_of_negations")
{
CHECK(R"(string & ~"world")" == toString(normal(R"(
(string & Not<"hello"> & Not<"world">) | (string & Not<"goodbye"> & Not<"world">)
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "disjoint_negations_normalize_to_string")
{
CHECK(R"(string)" == toString(normal(R"(
(string & Not<"hello"> & Not<"world">) | (string & Not<"goodbye">)
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negate_boolean")
{
CHECK("true" == toString(normal(R"(
boolean & Not<false>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negate_boolean_2")
{
CHECK("never" == toString(normal(R"(
true & Not<true>
)")));
}
Sync to upstream/release/566 (#853) * Fixed incorrect lexeme generated for string parts in the middle of an interpolated string (Fixes https://github.com/Roblox/luau/issues/744) * DeprecatedApi lint can report some issues without type inference information * Fixed performance of autocomplete requests when suggestions have large intersection types (Solves https://github.com/Roblox/luau/discussions/847) * Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC: Deprecate table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md)) * With -O2 optimization level, we now optimize builtin calls based on known argument/return count. Note that this change can be observable if `getfenv/setfenv` is used to substitute a builtin, especially if arity is different. Fastcall heavy tests show a 1-2% improvement. * Luau can now be built with clang-cl (Fixes https://github.com/Roblox/luau/issues/736) We also made many improvements to our experimental components. For our new type solver: * Overhauled data flow analysis system, fixed issues with 'repeat' loops, global variables and type annotations * Type refinements now work on generic table indexing with a string literal * Type refinements will properly track potentially 'nil' values (like t[x] for a missing key) and their further refinements * Internal top table type is now isomorphic to `{}` which fixes issues when `typeof(v) == 'table'` type refinement is handled * References to non-existent types in type annotations no longer resolve to 'error' type like in old solver * Improved handling of class unions in property access expressions * Fixed default type packs * Unsealed tables can now have metatables * Restored expected types for function arguments And for native code generation: * Added min and max IR instructions mapping to vminsd/vmaxsd on x64 * We now speculatively extract direct execution fast-paths based on expected types of expressions which provides better optimization opportunities inside a single basic block * Translated existing math fastcalls to IR form to improve tag guard removal and constant propagation
2023-03-04 04:21:14 +08:00
TEST_CASE_FIXTURE(NormalizeFixture, "double_negation")
{
CHECK("number" == toString(normal(R"(
number & Not<Not<any>>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negate_any")
{
CHECK("number" == toString(normal(R"(
number & Not<any>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "intersect_function_and_top_function")
{
CHECK("() -> ()" == toString(normal(R"(
fun & (() -> ())
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "intersect_function_and_top_function_reverse")
{
CHECK("() -> ()" == toString(normal(R"(
(() -> ()) & fun
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "union_function_and_top_function")
{
CHECK("function" == toString(normal(R"(
fun | (() -> ())
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negated_function_is_anything_except_a_function")
{
Sync to upstream/release/603 (#1097) # What's changed? - Record the location of properties for table types (closes #802) - Implement stricter UTF-8 validations as per the RFC (https://github.com/luau-lang/rfcs/pull/1) - Implement `buffer` as a new type in both the old and new solvers. - Changed errors produced by some `buffer` builtins to be a bit more generic to avoid platform-dependent error messages. - Fixed a bug where `Unifier` would copy some persistent types, tripping some internal assertions. - Type checking rules on relational operators is now a little bit more lax. - Improve dead code elimination for some `if` statements with complex always-false conditions ## New type solver - Dataflow analysis now generates phi nodes on exit of branches. - Dataflow analysis avoids producing a new definition for locals or properties that are not owned by that loop. - If a function parameter has been constrained to `never`, report errors at all uses of that parameter within that function. - Switch to using the new `Luau::Set` to replace `std::unordered_set` to alleviate some poor allocation characteristics which was negatively affecting overall performance. - Subtyping can now report many failing reasons instead of just the first one that we happened to find during the test. - Subtyping now also report reasons for type pack mismatches. - When visiting `if` statements or expressions, the resulting context are the common terms in both branches. ## Native codegen - Implement support for `buffer` builtins to its IR for x64 and A64. - Optimized `table.insert` by not inserting a table barrier if it is fastcalled with a constant. ## Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Arseny Kapoulkine <arseny@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-11 05:10:07 +08:00
CHECK("(boolean | buffer | class | number | string | table | thread)?" == toString(normal(R"(
Not<fun>
)")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "specific_functions_cannot_be_negated")
{
CHECK(nullptr == toNormalizedType("Not<(boolean) -> boolean>"));
}
TEST_CASE_FIXTURE(NormalizeFixture, "trivial_intersection_inhabited")
{
// this test was used to fix a bug in normalization when working with intersections/unions of the same type.
TypeId a = arena.addType(FunctionType{builtinTypes->emptyTypePack, builtinTypes->anyTypePack, std::nullopt, false});
TypeId c = arena.addType(IntersectionType{{a, a}});
std::shared_ptr<const NormalizedType> n = normalizer.normalize(c);
REQUIRE(n);
CHECK(normalizer.isInhabited(n.get()) == NormalizationResult::True);
}
TEST_CASE_FIXTURE(NormalizeFixture, "bare_negated_boolean")
{
Sync to upstream/release/603 (#1097) # What's changed? - Record the location of properties for table types (closes #802) - Implement stricter UTF-8 validations as per the RFC (https://github.com/luau-lang/rfcs/pull/1) - Implement `buffer` as a new type in both the old and new solvers. - Changed errors produced by some `buffer` builtins to be a bit more generic to avoid platform-dependent error messages. - Fixed a bug where `Unifier` would copy some persistent types, tripping some internal assertions. - Type checking rules on relational operators is now a little bit more lax. - Improve dead code elimination for some `if` statements with complex always-false conditions ## New type solver - Dataflow analysis now generates phi nodes on exit of branches. - Dataflow analysis avoids producing a new definition for locals or properties that are not owned by that loop. - If a function parameter has been constrained to `never`, report errors at all uses of that parameter within that function. - Switch to using the new `Luau::Set` to replace `std::unordered_set` to alleviate some poor allocation characteristics which was negatively affecting overall performance. - Subtyping can now report many failing reasons instead of just the first one that we happened to find during the test. - Subtyping now also report reasons for type pack mismatches. - When visiting `if` statements or expressions, the resulting context are the common terms in both branches. ## Native codegen - Implement support for `buffer` builtins to its IR for x64 and A64. - Optimized `table.insert` by not inserting a table barrier if it is fastcalled with a constant. ## Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Arseny Kapoulkine <arseny@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-11 05:10:07 +08:00
CHECK("(buffer | class | function | number | string | table | thread)?" == toString(normal(R"(
Not<boolean>
)")));
}
2022-04-15 07:57:43 +08:00
TEST_CASE_FIXTURE(Fixture, "higher_order_function")
{
check(R"(
function apply(f, x)
return f(x)
end
local a = apply(function(x: number) return x + x end, 5)
)");
TypeId aType = requireType("a");
CHECK_MESSAGE(isNumber(follow(aType)), "Expected a number but got ", toString(aType));
}
TEST_CASE_FIXTURE(Fixture, "higher_order_function_with_annotation")
{
check(R"(
function apply<a, b>(f: (a) -> b, x)
return f(x)
end
)");
CHECK_EQ("<a, b>((a) -> b, a) -> b", toString(requireType("apply")));
}
TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly")
{
CheckResult result = check(R"(
local Cyclic = {}
function Cyclic.get()
return Cyclic
end
)");
LUAU_REQUIRE_NO_ERRORS(result);
TypeId ty = requireType("Cyclic");
CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true}));
}
TEST_CASE_FIXTURE(BuiltinsFixture, "skip_force_normal_on_external_types")
{
createSomeClasses(&frontend);
CheckResult result = check(R"(
export type t0 = { a: Child }
export type t1 = { a: typeof(string.byte) }
)");
LUAU_REQUIRE_NO_ERRORS(result);
}
TEST_CASE_FIXTURE(Fixture, "intersection_combine_on_bound_self")
{
CheckResult result = check(R"(
export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))
)");
LUAU_REQUIRE_ERRORS(result);
}
TEST_CASE_FIXTURE(NormalizeFixture, "unions_of_classes")
{
createSomeClasses(&frontend);
CHECK("Parent | Unrelated" == toString(normal("Parent | Unrelated")));
CHECK("Parent" == toString(normal("Parent | Child")));
CHECK("Parent | Unrelated" == toString(normal("Parent | Child | Unrelated")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "intersections_of_classes")
{
createSomeClasses(&frontend);
CHECK("Child" == toString(normal("Parent & Child")));
CHECK("never" == toString(normal("Child & Unrelated")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "narrow_union_of_classes_with_intersection")
{
createSomeClasses(&frontend);
CHECK("Child" == toString(normal("(Child | Unrelated) & Child")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_metatables_where_the_metatable_is_top_or_bottom")
{
if (FFlag::DebugLuauDeferredConstraintResolution)
CHECK("{ @metatable *error-type*, { } }" == toString(normal("Mt<{}, any> & Mt<{}, err>")));
else
CHECK("{ @metatable *error-type*, {| |} }" == toString(normal("Mt<{}, any> & Mt<{}, err>")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "recurring_intersection")
{
CheckResult result = check(R"(
type A = any?
type B = A & A
)");
std::optional<TypeId> t = lookupType("B");
REQUIRE(t);
std::shared_ptr<const NormalizedType> nt = normalizer.normalize(*t);
REQUIRE(nt);
CHECK("any" == toString(normalizer.typeFromNormal(*nt)));
}
TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_union")
{
// T where T = any & (number | T)
TypeId t = arena.addType(BlockedType{});
TypeId u = arena.addType(UnionType{{builtinTypes->numberType, t}});
asMutable(t)->ty.emplace<IntersectionType>(IntersectionType{{builtinTypes->anyType, u}});
std::shared_ptr<const NormalizedType> nt = normalizer.normalize(t);
REQUIRE(nt);
CHECK("number" == toString(normalizer.typeFromNormal(*nt)));
}
Sync to upstream/release/622 (#1232) # What's changed? * Improved the actual message for the type errors for `cannot call non-function` when attempting to call a union of functions/callable tables. The error now correctly explains the issue is an inability to determine the return type of the call in this situation. * Resolve an issue where tables and metatables were not correctly being cloned during instantiation (fixes #1176). * Refactor `luaM_getnextgcopage` to `luaM_getnextpage` (generally removing `gco` prefix where appropriate). * Optimize `table.move` between tables for large ranges of elements. * Reformat a bunch of code automatically using `clang-format`. ### New Type Solver * Clean up minimally-used or unused constraints in the constraint solver (`InstantiationConstraint`, `SetOpConstraint`, `SingletonOrTopTypeConstraint`). * Add a builtin `singleton` type family to replace `SingletonOrTopTypeConstraint` when inferring refinements. * Fixed a crash involving type path reasoning by recording when type family reduction has taken place in the path. * Improved constraint ordering by blocking on unreduced types families that are not yet proven uninhabitable. * Improved the handling of `SetIndexerConstraints` for both better inference quality and to resolve crashes. * Fix a crash when normalizing cyclic unions of intersections. * Fix a crash when normalizing an intersection with the negation of `unknown`. * Fix a number of crashes caused by missing `follow` calls on `TypeId`s. * Changed type family reduction to correctly use a semantic notion of uninhabited types, rather than checking for `never` types specifically. * Refactor the `union` and `intersect` type families to be variadic. ### Native Code Generation * Improve translation for userdata key get/set and userdata/vector namecall. * Provide `[top level]` and `[anonymous]` as function names to `FunctionStats` as appropriate when no function name is available. * Disable unwind support on Android platforms since it is unsupported. * --- ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-04-20 05:48:02 +08:00
TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_union_of_intersection")
{
// t1 where t1 = (string & t1) | string
TypeId boundTy = arena.addType(BlockedType{});
TypeId intersectTy = arena.addType(IntersectionType{{builtinTypes->stringType, boundTy}});
TypeId unionTy = arena.addType(UnionType{{builtinTypes->stringType, intersectTy}});
asMutable(boundTy)->reassign(Type{BoundType{unionTy}});
std::shared_ptr<const NormalizedType> nt = normalizer.normalize(unionTy);
CHECK("string" == toString(normalizer.typeFromNormal(*nt)));
}
TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_intersection_of_unions")
{
// t1 where t1 = (string & t1) | string
TypeId boundTy = arena.addType(BlockedType{});
TypeId unionTy = arena.addType(UnionType{{builtinTypes->stringType, boundTy}});
TypeId intersectionTy = arena.addType(IntersectionType{{builtinTypes->stringType, unionTy}});
asMutable(boundTy)->reassign(Type{BoundType{intersectionTy}});
std::shared_ptr<const NormalizedType> nt = normalizer.normalize(intersectionTy);
CHECK("string" == toString(normalizer.typeFromNormal(*nt)));
}
TEST_CASE_FIXTURE(NormalizeFixture, "crazy_metatable")
{
CHECK("never" == toString(normal("Mt<{}, number> & Mt<{}, string>")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negations_of_classes")
{
createSomeClasses(&frontend);
CHECK("(Parent & ~Child) | Unrelated" == toString(normal("(Parent & Not<Child>) | Unrelated")));
Sync to upstream/release/603 (#1097) # What's changed? - Record the location of properties for table types (closes #802) - Implement stricter UTF-8 validations as per the RFC (https://github.com/luau-lang/rfcs/pull/1) - Implement `buffer` as a new type in both the old and new solvers. - Changed errors produced by some `buffer` builtins to be a bit more generic to avoid platform-dependent error messages. - Fixed a bug where `Unifier` would copy some persistent types, tripping some internal assertions. - Type checking rules on relational operators is now a little bit more lax. - Improve dead code elimination for some `if` statements with complex always-false conditions ## New type solver - Dataflow analysis now generates phi nodes on exit of branches. - Dataflow analysis avoids producing a new definition for locals or properties that are not owned by that loop. - If a function parameter has been constrained to `never`, report errors at all uses of that parameter within that function. - Switch to using the new `Luau::Set` to replace `std::unordered_set` to alleviate some poor allocation characteristics which was negatively affecting overall performance. - Subtyping can now report many failing reasons instead of just the first one that we happened to find during the test. - Subtyping now also report reasons for type pack mismatches. - When visiting `if` statements or expressions, the resulting context are the common terms in both branches. ## Native codegen - Implement support for `buffer` builtins to its IR for x64 and A64. - Optimized `table.insert` by not inserting a table barrier if it is fastcalled with a constant. ## Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Arseny Kapoulkine <arseny@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-11 05:10:07 +08:00
CHECK("((class & ~Child) | boolean | buffer | function | number | string | table | thread)?" == toString(normal("Not<Child>")));
CHECK("Child" == toString(normal("Not<Parent> & Child")));
Sync to upstream/release/603 (#1097) # What's changed? - Record the location of properties for table types (closes #802) - Implement stricter UTF-8 validations as per the RFC (https://github.com/luau-lang/rfcs/pull/1) - Implement `buffer` as a new type in both the old and new solvers. - Changed errors produced by some `buffer` builtins to be a bit more generic to avoid platform-dependent error messages. - Fixed a bug where `Unifier` would copy some persistent types, tripping some internal assertions. - Type checking rules on relational operators is now a little bit more lax. - Improve dead code elimination for some `if` statements with complex always-false conditions ## New type solver - Dataflow analysis now generates phi nodes on exit of branches. - Dataflow analysis avoids producing a new definition for locals or properties that are not owned by that loop. - If a function parameter has been constrained to `never`, report errors at all uses of that parameter within that function. - Switch to using the new `Luau::Set` to replace `std::unordered_set` to alleviate some poor allocation characteristics which was negatively affecting overall performance. - Subtyping can now report many failing reasons instead of just the first one that we happened to find during the test. - Subtyping now also report reasons for type pack mismatches. - When visiting `if` statements or expressions, the resulting context are the common terms in both branches. ## Native codegen - Implement support for `buffer` builtins to its IR for x64 and A64. - Optimized `table.insert` by not inserting a table barrier if it is fastcalled with a constant. ## Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Arseny Kapoulkine <arseny@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-11 05:10:07 +08:00
CHECK("((class & ~Parent) | Child | boolean | buffer | function | number | string | table | thread)?" == toString(normal("Not<Parent> | Child")));
CHECK("(boolean | buffer | function | number | string | table | thread)?" == toString(normal("Not<cls>")));
CHECK("(Parent | Unrelated | boolean | buffer | function | number | string | table | thread)?" ==
toString(normal("Not<cls & Not<Parent> & Not<Child> & Not<Unrelated>>")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "classes_and_unknown")
{
createSomeClasses(&frontend);
CHECK("Parent" == toString(normal("Parent & unknown")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "classes_and_never")
{
createSomeClasses(&frontend);
CHECK("never" == toString(normal("Parent & never")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "top_table_type")
{
CHECK("table" == toString(normal("{} | tbl")));
if (FFlag::DebugLuauDeferredConstraintResolution)
CHECK("{ }" == toString(normal("{} & tbl")));
else
CHECK("{| |}" == toString(normal("{} & tbl")));
CHECK("never" == toString(normal("number & tbl")));
}
TEST_CASE_FIXTURE(NormalizeFixture, "negations_of_tables")
{
CHECK(nullptr == toNormalizedType("Not<{}>"));
Sync to upstream/release/603 (#1097) # What's changed? - Record the location of properties for table types (closes #802) - Implement stricter UTF-8 validations as per the RFC (https://github.com/luau-lang/rfcs/pull/1) - Implement `buffer` as a new type in both the old and new solvers. - Changed errors produced by some `buffer` builtins to be a bit more generic to avoid platform-dependent error messages. - Fixed a bug where `Unifier` would copy some persistent types, tripping some internal assertions. - Type checking rules on relational operators is now a little bit more lax. - Improve dead code elimination for some `if` statements with complex always-false conditions ## New type solver - Dataflow analysis now generates phi nodes on exit of branches. - Dataflow analysis avoids producing a new definition for locals or properties that are not owned by that loop. - If a function parameter has been constrained to `never`, report errors at all uses of that parameter within that function. - Switch to using the new `Luau::Set` to replace `std::unordered_set` to alleviate some poor allocation characteristics which was negatively affecting overall performance. - Subtyping can now report many failing reasons instead of just the first one that we happened to find during the test. - Subtyping now also report reasons for type pack mismatches. - When visiting `if` statements or expressions, the resulting context are the common terms in both branches. ## Native codegen - Implement support for `buffer` builtins to its IR for x64 and A64. - Optimized `table.insert` by not inserting a table barrier if it is fastcalled with a constant. ## Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Arseny Kapoulkine <arseny@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-11 05:10:07 +08:00
CHECK("(boolean | buffer | class | function | number | string | thread)?" == toString(normal("Not<tbl>")));
CHECK("table" == toString(normal("Not<Not<tbl>>")));
}
Sync to upstream/release/568 (#865) * A small subset of control-flow refinements have been added to recognize type options that are unreachable after a conditional/unconditional code block. (Fixes https://github.com/Roblox/luau/issues/356). Some examples: ```lua local function f(x: string?) if not x then return end -- x is 'string' here end ``` Throwing calls like `error` or `assert(false)` instead of 'return' are also recognized. Existing complex refinements like type/typeof and tagged union checks are expected to work, among others. To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to be removed from `ExperimentalFlags.h`. If will become enabled unconditionally in the near future. * Linter has been integrated into the typechecker analysis so that type-aware lint warnings can work in any mode `Frontend::lint` methods were deprecated, `Frontend::check` has to be used instead with `runLintChecks` option set. Resulting lint warning are located inside `CheckResult`. * Fixed large performance drop and increased memory consumption when array is filled at an offset (Fixes https://github.com/Roblox/luau/issues/590) * Part of [Type error suppression RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md) was implemented making subtyping checks with `any` type transitive. --- In our work on the new type-solver: * `--!nocheck` mode no longer reports type errors * New solver will not be used for `--!nonstrict` modules until all issues with strict mode typechecking are fixed * Added control-flow aware type refinements mentioned earlier In native code generation: * `LOP_NAMECALL` has been translated to IR * `type` and `typeof` builtin fastcalls have been translated to IR/assembly * Additional steps were taken towards arm64 support
2023-03-18 03:20:37 +08:00
TEST_CASE_FIXTURE(NormalizeFixture, "normalize_blocked_types")
{
Type blocked{BlockedType{}};
std::shared_ptr<const NormalizedType> norm = normalizer.normalize(&blocked);
Sync to upstream/release/568 (#865) * A small subset of control-flow refinements have been added to recognize type options that are unreachable after a conditional/unconditional code block. (Fixes https://github.com/Roblox/luau/issues/356). Some examples: ```lua local function f(x: string?) if not x then return end -- x is 'string' here end ``` Throwing calls like `error` or `assert(false)` instead of 'return' are also recognized. Existing complex refinements like type/typeof and tagged union checks are expected to work, among others. To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to be removed from `ExperimentalFlags.h`. If will become enabled unconditionally in the near future. * Linter has been integrated into the typechecker analysis so that type-aware lint warnings can work in any mode `Frontend::lint` methods were deprecated, `Frontend::check` has to be used instead with `runLintChecks` option set. Resulting lint warning are located inside `CheckResult`. * Fixed large performance drop and increased memory consumption when array is filled at an offset (Fixes https://github.com/Roblox/luau/issues/590) * Part of [Type error suppression RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md) was implemented making subtyping checks with `any` type transitive. --- In our work on the new type-solver: * `--!nocheck` mode no longer reports type errors * New solver will not be used for `--!nonstrict` modules until all issues with strict mode typechecking are fixed * Added control-flow aware type refinements mentioned earlier In native code generation: * `LOP_NAMECALL` has been translated to IR * `type` and `typeof` builtin fastcalls have been translated to IR/assembly * Additional steps were taken towards arm64 support
2023-03-18 03:20:37 +08:00
CHECK_EQ(normalizer.typeFromNormal(*norm), &blocked);
}
TEST_CASE_FIXTURE(NormalizeFixture, "normalize_is_exactly_number")
{
std::shared_ptr<const NormalizedType> number = normalizer.normalize(builtinTypes->numberType);
// 1. all types for which Types::number say true for, NormalizedType::isExactlyNumber should say true as well
CHECK(Luau::isNumber(builtinTypes->numberType) == number->isExactlyNumber());
// 2. isExactlyNumber should handle cases like `number & number`
TypeId intersection = arena.addType(IntersectionType{{builtinTypes->numberType, builtinTypes->numberType}});
std::shared_ptr<const NormalizedType> normIntersection = normalizer.normalize(intersection);
CHECK(normIntersection->isExactlyNumber());
// 3. isExactlyNumber should reject things that are definitely not precisely numbers `number | any`
TypeId yoonion = arena.addType(UnionType{{builtinTypes->anyType, builtinTypes->numberType}});
std::shared_ptr<const NormalizedType> unionIntersection = normalizer.normalize(yoonion);
CHECK(!unionIntersection->isExactlyNumber());
}
TEST_CASE_FIXTURE(NormalizeFixture, "normalize_unknown")
{
auto nt = toNormalizedType("Not<string> | Not<number>");
CHECK(nt);
CHECK(nt->isUnknown());
CHECK(toString(normalizer.typeFromNormal(*nt)) == "unknown");
}
Sync to upstream/release/614 (#1173) # What's changed? Add program argument passing to scripts run using the Luau REPL! You can now pass `--program-args` (or shorthand `-a`) to the REPL which will treat all remaining arguments as arguments to pass to executed scripts. These values can be accessed through variadic argument expansion. You can read these values like so: ``` local args = {...} -- gets you an array of all the arguments ``` For example if we run the following script like `luau test.lua -a test1 test2 test3`: ``` -- test.lua print(...) ``` you should get the output: ``` test1 test2 test3 ``` ### Native Code Generation * Improve A64 lowering for vector operations by using vector instructions * Fix lowering issue in IR value location tracking! - A developer reported a divergence between code run in the VM and Native Code Generation which we have now fixed ### New Type Solver * Apply substitution to type families, and emit new constraints to reduce those further * More progress on reducing comparison (`lt/le`)type families * Resolve two major sources of cyclic types in the new solver ### Miscellaneous * Turned internal compiler errors (ICE's) into warnings and errors ------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-02-24 04:08:34 +08:00
TEST_CASE_FIXTURE(NormalizeFixture, "read_only_props")
{
ScopedFastFlag sff{FFlag::DebugLuauDeferredConstraintResolution, true};
CHECK("{ x: string }" == toString(normal("{ read x: string } & { x: string }"), {true}));
CHECK("{ x: string }" == toString(normal("{ x: string } & { read x: string }"), {true}));
}
TEST_CASE_FIXTURE(NormalizeFixture, "read_only_props_2")
{
ScopedFastFlag sff{FFlag::DebugLuauDeferredConstraintResolution, true};
CHECK(R"({ x: "hello" })" == toString(normal(R"({ x: "hello" } & { x: string })"), {true}));
CHECK(R"(never)" == toString(normal(R"({ x: "hello" } & { x: "world" })"), {true}));
Sync to upstream/release/614 (#1173) # What's changed? Add program argument passing to scripts run using the Luau REPL! You can now pass `--program-args` (or shorthand `-a`) to the REPL which will treat all remaining arguments as arguments to pass to executed scripts. These values can be accessed through variadic argument expansion. You can read these values like so: ``` local args = {...} -- gets you an array of all the arguments ``` For example if we run the following script like `luau test.lua -a test1 test2 test3`: ``` -- test.lua print(...) ``` you should get the output: ``` test1 test2 test3 ``` ### Native Code Generation * Improve A64 lowering for vector operations by using vector instructions * Fix lowering issue in IR value location tracking! - A developer reported a divergence between code run in the VM and Native Code Generation which we have now fixed ### New Type Solver * Apply substitution to type families, and emit new constraints to reduce those further * More progress on reducing comparison (`lt/le`)type families * Resolve two major sources of cyclic types in the new solver ### Miscellaneous * Turned internal compiler errors (ICE's) into warnings and errors ------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-02-24 04:08:34 +08:00
}
TEST_CASE_FIXTURE(NormalizeFixture, "read_only_props_3")
{
ScopedFastFlag sff{FFlag::DebugLuauDeferredConstraintResolution, true};
CHECK(R"({ read x: "hello" })" == toString(normal(R"({ read x: "hello" } & { read x: string })"), {true}));
CHECK("never" == toString(normal(R"({ read x: "hello" } & { read x: "world" })"), {true}));
}
TEST_CASE_FIXTURE(NormalizeFixture, "final_types_are_cached")
{
std::shared_ptr<const NormalizedType> na1 = normalizer.normalize(builtinTypes->numberType);
std::shared_ptr<const NormalizedType> na2 = normalizer.normalize(builtinTypes->numberType);
CHECK(na1 == na2);
}
TEST_CASE_FIXTURE(NormalizeFixture, "non_final_types_can_be_normalized_but_are_not_cached")
{
TypeId a = arena.freshType(&globalScope);
std::shared_ptr<const NormalizedType> na1 = normalizer.normalize(a);
std::shared_ptr<const NormalizedType> na2 = normalizer.normalize(a);
CHECK(na1 != na2);
Sync to upstream/release/614 (#1173) # What's changed? Add program argument passing to scripts run using the Luau REPL! You can now pass `--program-args` (or shorthand `-a`) to the REPL which will treat all remaining arguments as arguments to pass to executed scripts. These values can be accessed through variadic argument expansion. You can read these values like so: ``` local args = {...} -- gets you an array of all the arguments ``` For example if we run the following script like `luau test.lua -a test1 test2 test3`: ``` -- test.lua print(...) ``` you should get the output: ``` test1 test2 test3 ``` ### Native Code Generation * Improve A64 lowering for vector operations by using vector instructions * Fix lowering issue in IR value location tracking! - A developer reported a divergence between code run in the VM and Native Code Generation which we have now fixed ### New Type Solver * Apply substitution to type families, and emit new constraints to reduce those further * More progress on reducing comparison (`lt/le`)type families * Resolve two major sources of cyclic types in the new solver ### Miscellaneous * Turned internal compiler errors (ICE's) into warnings and errors ------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-02-24 04:08:34 +08:00
}
Sync to upstream/release/622 (#1232) # What's changed? * Improved the actual message for the type errors for `cannot call non-function` when attempting to call a union of functions/callable tables. The error now correctly explains the issue is an inability to determine the return type of the call in this situation. * Resolve an issue where tables and metatables were not correctly being cloned during instantiation (fixes #1176). * Refactor `luaM_getnextgcopage` to `luaM_getnextpage` (generally removing `gco` prefix where appropriate). * Optimize `table.move` between tables for large ranges of elements. * Reformat a bunch of code automatically using `clang-format`. ### New Type Solver * Clean up minimally-used or unused constraints in the constraint solver (`InstantiationConstraint`, `SetOpConstraint`, `SingletonOrTopTypeConstraint`). * Add a builtin `singleton` type family to replace `SingletonOrTopTypeConstraint` when inferring refinements. * Fixed a crash involving type path reasoning by recording when type family reduction has taken place in the path. * Improved constraint ordering by blocking on unreduced types families that are not yet proven uninhabitable. * Improved the handling of `SetIndexerConstraints` for both better inference quality and to resolve crashes. * Fix a crash when normalizing cyclic unions of intersections. * Fix a crash when normalizing an intersection with the negation of `unknown`. * Fix a number of crashes caused by missing `follow` calls on `TypeId`s. * Changed type family reduction to correctly use a semantic notion of uninhabited types, rather than checking for `never` types specifically. * Refactor the `union` and `intersect` type families to be variadic. ### Native Code Generation * Improve translation for userdata key get/set and userdata/vector namecall. * Provide `[top level]` and `[anonymous]` as function names to `FunctionStats` as appropriate when no function name is available. * Disable unwind support on Android platforms since it is unsupported. * --- ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-04-20 05:48:02 +08:00
TEST_CASE_FIXTURE(NormalizeFixture, "intersect_with_not_unknown")
{
ScopedFastFlag sff{FFlag::LuauNormalizeNotUnknownIntersection, true};
TypeId notUnknown = arena.addType(NegationType{builtinTypes->unknownType});
TypeId type = arena.addType(IntersectionType{{builtinTypes->numberType, notUnknown}});
std::shared_ptr<const NormalizedType> normalized = normalizer.normalize(type);
CHECK("never" == toString(normalizer.typeFromNormal(*normalized.get())));
}
TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_stack_overflow_1")
{
ScopedFastInt sfi{FInt::LuauTypeInferRecursionLimit, 165};
this->unifierState.counters.recursionLimit = FInt::LuauTypeInferRecursionLimit;
TypeId t1 = arena.addType(TableType{});
TypeId t2 = arena.addType(TableType{});
TypeId t3 = arena.addType(IntersectionType{{t1, t2}});
asMutable(t1)->ty.get_if<TableType>()->props = {{"foo", Property::readonly(t2)}};
asMutable(t2)->ty.get_if<TableType>()->props = {{"foo", Property::readonly(t1)}};
std::shared_ptr<const NormalizedType> normalized = normalizer.normalize(t3);
CHECK(normalized);
}
TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_stack_overflow_2")
{
ScopedFastInt sfi{FInt::LuauTypeInferRecursionLimit, 165};
this->unifierState.counters.recursionLimit = FInt::LuauTypeInferRecursionLimit;
TypeId t1 = arena.addType(TableType{});
TypeId t2 = arena.addType(TableType{});
TypeId t3 = arena.addType(IntersectionType{{t1, t2}});
asMutable(t1)->ty.get_if<TableType>()->props = {{"foo", Property::readonly(t3)}};
asMutable(t2)->ty.get_if<TableType>()->props = {{"foo", Property::readonly(t1)}};
std::shared_ptr<const NormalizedType> normalized = normalizer.normalize(t3);
CHECK(normalized);
}
2022-04-15 07:57:43 +08:00
TEST_SUITE_END();