fix(core): Fix data transformation function that are reported not to work properly (#5338)

* 🔥 Remove test extensions * 🚧 Add test description * 📘 Expand types * ⚡ Export extensions * ⚡ Export collection * ⚡ Mark all proxies * ✏️ Rename for clarity * ⚡ Export from barrel * ✨ Create datatype completions * ⚡ Mount datatype completions * 🧪 Adjust tests * ⚡ Add `path` prop * 🔥 Remove `()` from completion labels * ⚡ Filter out completions for pseudo-proxies * 🐛 Fix method error * ⚡ Add metrics * ✏️ Improve naming * ✨ Start completion on empty resolvable * ✨ Implement completion previews * ⚡ Break out completion manager * ⚡ Implement in expression editor modal * ✏️ Improve naming * ⚡ Filter out irrelevant completions * ✨ Add preview hint * ✏️ Improve comments * 🎨 Style preview hint * ⚡ Expand `hasNoParams` * ⚡ Add spacing for readability * ⚡ Add error codes * ✏️ Add comment * 🐛 Fix Esc behavior * ⚡ Parse Unicode * ⚡ Throw on invalid `DateTime` * ⚡ Fix second root completion detection * ⚡ Switch message at completable prefix position * 🐛 Fix function names for non-dev build * 🐛 Fix `json` handling * 🔥 Comment out previews * ♻️ Apply feedback * 🔥 Remove extensions * 🚚 Rename extensions * ⚡ Adjust some implementations * 🔥 Remove dummy extensions * 🐛 Fix object regex * ♻️ Apply feedback * ✏️ Fix typos * ✏️ Add `fn is not a function` message * 🔥 Remove check * ✨ Add `isNotEmpty` for objects * 🚚 Rename `global` to `alpha` * 🔥 Remove `encrypt` * ⚡ Restore `is not a function` error * ⚡ Support `week` on `extract()` * 🧪 Fix tests * ⚡ Add validation to some string extensions * ⚡ Validate number arrays in some extensions * 🧪 Fix tests * ✏️ Improve error message * ⏪ Revert extensions framework changes * 🧹 Previews cleanup * ⚡ Condense blank completions * ⚡ Refactor dollar completions * ⚡ Refactor non-dollar completions * ⚡ Refactor Luxon completions * ⚡ Refactor datatype completions * ⚡ Use `DATETIMEUNIT_MAP` * ✏️ Update test description * ⏪ Revert "Use `DATETIMEUNIT_MAP`" This reverts commit 472a77df5cd789905d162f3c3db02ac767b89b4e. * 🧪 Add tests * ♻️ Restore generic extensions * 🔥 Remove logs * 🧪 Expand tests * ✨ Add `Math` completions * ✏️ List breaking change * ⚡ Add doc tooltips * 🐛 Fix node selector regex * 🐛 Fix `context` resolution * 🐛 Allow dollar completions in args * ⚡ Make numeric array methods context-dependent * 📝 Adjust docs * 🐛 Fix selector ref * ⚡ Surface error for valid URL * 🐛 Disallow whitespace in `isEmail` check * 🧪 Fix test for `isUrl` * ⚡ Add comma validator in `toFloat` * ⚡ Add validation to `$jmespath()` * ⏪ Revert valid URL error * ⚡ Adjust `$jmespath()` validation * 🧪 Adjust `isUrl` test * ⚡ Remove `{}` and `[]` from compact * ✏️ Update docs * 🚚 Rename `stripTags` to `removeTags` * ⚡ Do not inject whitespace inside resolvable * ⚡ Make completions aware of `()` * ✏️ Add note * ⚡ Update sorting * ⚡ Hide active node name from node selector * 🔥 Remove `length()` and its aliases * ⚡ Validate non-zero for `chunk` * ✏️ Reword all error messages * 🐛 Fix `$now` and `$today` * ⚡ Simplify with `stripExcessParens` * ⚡ Fold luxon into datatype * 🧪 Clean up tests * 🔥 Remove tests for removed methods * 👕 Fix type * ⬆️ Upgrade lang pack * ⏪ Undo change to `vitest` command * 🔥 Remove unused method * ⚡ Separate `return` line * ✏️ Improve description * 🧪 Expand tests for initial-only completions * 🧪 Add bracket-aware completions * ⚡ Make check for `all()` stricter * ✏️ Adjust explanatory comments * 🔥 Remove unneded copy * 🔥 Remove outdated comment * ⚡ Make naming consistent * ✏️ Update comments * ⚡ Improve URL scheme check * ✏️ Add comment * 🚚 Move extension * ✏️ Update `BREAKING-CHANGES.md` * ✏️ Update upcoming version * ✏️ Fix grammar * ✏️ Shorten message * 🐛 Fix `Esc` behavior * 🐛 Fix `isNumeric` * ✨ Using UTC to handle-dates on back-end * ✅ Added more unit tests for date extensions * ⚡ Not using `JSON.stringify` to render dates * ⚡ Using `deep-equal` library instead of our `deepCompare` function * ✅ Adding more tests to array extensions * ⚡ Fixing `inBetween` extension function * ✅ Added tests for `.inBetween()` * ⚡ Updating `isEven` and `isOdd` to throw for floats * ⚡ Updating `Array.merge()` so it works without arguments * 🔀 Fixing leftover merge confilct * ⚡ Updating `removeFieldsContaining` and `keepFieldsContaining` to throw on empty strings * ⚡ Fixing `pluck()` so it returns only plucked values * ⬆️ Updating pnpm lockfile * 👕 Fixing lint errors * ⚡ Using workflow timezone to display dates * ✔️ Updating tests with workflow timezone * ⚡ Not using system timezone when creating Luxon dates * ⚡ Updating `merge()` and `pluck()` array functions * 🔀 Sync with `master`: Removing code that was preserved during merge * ⚡ Updating `.pluck()` to return full array if no arguments are passed * ⚡ Updating `keepFieldsContaining` and `merge` object functions * ⚡ Using week as default for `date.extract()` * ✅ Adding more test cases for DT functions * ⚡ Removing `Object.merge` extension function. Adding missing `deep-equal` dependency * ⚡ Handling `toDate` case when time component is not specified * ⚡ Using workflow's timezone to render dates in output panel, updated unit tests after removing `Object.merge` function * ⚡ Not parsing numbers as dates * 👕 Fixing lint errors * ⚡ Fixing a typo * ⚡ Making date detection more strict so only stringified dates are getting converted * 👌 Addressing PR feedback * 🔥 Removing leftover comment --------- Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2025-12-17 10:02:05 +00:00 · 2023-02-15 10:50:16 +01:00
parent 83505cb0d4
commit 0cf45bc4c8
17 changed files with 372 additions and 224 deletions
--- a/packages/workflow/test/ExpressionExtensions/ArrayExtensions.test.ts
+++ b/packages/workflow/test/ExpressionExtensions/ArrayExtensions.test.ts
@@ -22,16 +22,58 @@ describe('Data Transformation Functions', () => {
 				{ value: 3, string: '3' },
 				{ value: 4, string: '4' },
 				{ value: 5, string: '5' },
-				{ value: 6, string: '6' }
+				{ value: 6, string: '6' },
+				{ value: { something: 'else' } }
 			].pluck("value") }}`),
+			).toEqual(
+				expect.arrayContaining([1, 2, 3, 4, 5, 6, { something: 'else' }]),
+			);
+		});
+
+		test('.pluck() should work correctly for multiple values', () => {
+			expect(
+				evaluate(`={{ [
+					{
+						firstName: 'John',
+						lastName: 'Doe',
+						phone: {
+							home: '111-222',
+							office: '333-444'
+						}
+					},
+					{
+						firstName: 'Jane',
+						lastName: 'Doe',
+						phone: {
+							office: '555-666'
+						}
+					}
+			].pluck("firstName", "lastName") }}`),
+			).toEqual(
+				expect.arrayContaining([["John", "Doe"],["Jane", "Doe"]]),
+			);
+		});
+
+		test('.pluck() should work return everything with no args', () => {
+			expect(
+				evaluate(`={{ [
+				{ value: 1, string: '1' },
+				{ value: 2, string: '2' },
+				{ value: 3, string: '3' },
+				{ value: 4, string: '4' },
+				{ value: 5, string: '5' },
+				{ value: 6, string: '6' },
+				{ value: { something: 'else' } }
+			].pluck() }}`),
 			).toEqual(
 				expect.arrayContaining([
-					{ value: 1 },
-					{ value: 2 },
-					{ value: 3 },
-					{ value: 4 },
-					{ value: 5 },
-					{ value: 6 },
+					{ value: 1, string: '1' },
+					{ value: 2, string: '2' },
+					{ value: 3, string: '3' },
+					{ value: 4, string: '4' },
+					{ value: 5, string: '5' },
+					{ value: 6, string: '6' },
+					{ value: { something: 'else' } }
 				]),
 			);
 		});
@@ -42,6 +84,14 @@ describe('Data Transformation Functions', () => {
 			);
 		});

+		test('.unique() should work on an arrays containing nulls, objects and arrays', () => {
+			expect(
+				evaluate(
+					'={{ [1, 2, 3, "as", {}, {}, 1, 2, [1,2], "[sad]", "[sad]", null].unique() }}',
+				),
+			).toEqual([1, 2, 3, "as", {}, [1,2], "[sad]", null]);
+		});
+
 		test('.isEmpty() should work correctly on an array', () => {
 			expect(evaluate('={{ [].isEmpty() }}')).toEqual(true);
 		});
@@ -63,10 +113,15 @@ describe('Data Transformation Functions', () => {
 				evaluate(
 					'={{ [{ test1: 1, test2: 2 }, { test1: 1, test3: 3 }].merge([{ test1: 2, test3: 3 }, { test4: 4 }]) }}',
 				),
-			).toEqual([
-				{ test1: 1, test2: 2, test3: 3 },
-				{ test1: 1, test3: 3, test4: 4 },
-			]);
+			).toEqual({"test1": 1, "test2": 2, "test3": 3, "test4": 4});
+		});
+
+		test('.merge() should work correctly without arguments', () => {
+			expect(
+				evaluate(
+					'={{ [{ a: 1, some: null }, { a: 2, c: "something" }, 2, "asds", { b: 23 }, null, [1, 2]].merge() }}',
+				),
+			).toEqual({"a": 1, "some": null, "c": "something", "b": 23});
 		});

 		test('.smartJoin() should work correctly on an array of objects', () => {
@@ -119,6 +174,14 @@ describe('Data Transformation Functions', () => {
 			).toEqual([{ test1: 1 }, { test2: 2 }, { test1: 1, test3: 3 }, { test4: 4 }]);
 		});

+		test('.union() should work on an arrays containing nulls, objects and arrays', () => {
+			expect(
+				evaluate(
+					'={{ [1, 2, "dd", {}, null].union([1, {}, null, 3]) }}',
+				),
+			).toEqual([1, 2, "dd", {}, null, 3]);
+		});
+
 		test('.intersection() should work on an array of objects', () => {
 			expect(
 				evaluate(
@@ -127,6 +190,14 @@ describe('Data Transformation Functions', () => {
 			).toEqual([{ test2: 2 }]);
 		});

+		test('.intersection() should work on an arrays containing nulls, objects and arrays', () => {
+			expect(
+				evaluate(
+					'={{ [1, 2, "dd", {}, null].intersection([1, {}, null]) }}',
+				),
+			).toEqual([1, {}, null]);
+		});
+
 		test('.difference() should work on an array of objects', () => {
 			expect(
 				evaluate(
@@ -139,6 +210,14 @@ describe('Data Transformation Functions', () => {
 			).toEqual([]);
 		});

+		test('.difference() should work on an arrays containing nulls, objects and arrays', () => {
+			expect(
+				evaluate(
+					'={{ [1, 2, "dd", {}, null, ["a", 1]].difference([1, {}, null, ["a", 1]]) }}',
+				),
+			).toEqual([2, "dd"]);
+		});
+
 		test('.compact() should work on an array', () => {
 			expect(
 				evaluate(
--- a/packages/workflow/test/ExpressionExtensions/DateExtensions.test.ts
+++ b/packages/workflow/test/ExpressionExtensions/DateExtensions.test.ts
@@ -20,6 +20,33 @@ describe('Data Transformation Functions', () => {
 			);
 		});

+		test('.beginningOf("week") should work correctly on a string', () => {
+			const evaluatedDate = evaluate('={{ "2023-01-30".toDate().beginningOf("week") }}');
+			const expectedDate = DateTime.local(2023, 1, 23, { zone: TEST_TIMEZONE }).toJSDate();
+
+			if (evaluatedDate && evaluatedDate instanceof Date) {
+				expect(evaluatedDate.toDateString()).toEqual(expectedDate.toDateString());
+			}
+		});
+
+		test('.beginningOf("month") should work correctly on a string', () => {
+			const evaluatedDate = evaluate('={{ "2023-06-16".toDate().beginningOf("month") }}');
+			const expectedDate = DateTime.local(2023, 6, 1, { zone: TEST_TIMEZONE }).toJSDate();
+
+			if (evaluatedDate && evaluatedDate instanceof Date) {
+				expect(evaluatedDate.toDateString()).toEqual(expectedDate.toDateString());
+			}
+		});
+
+		test('.beginningOf("year") should work correctly on a string', () => {
+			const evaluatedDate = evaluate('={{ "2023-01-30".toDate().beginningOf("year") }}');
+			const expectedDate = DateTime.local(2023, 1, 1, { zone: TEST_TIMEZONE }).toJSDate();
+
+			if (evaluatedDate && evaluatedDate instanceof Date) {
+				expect(evaluatedDate.toDateString()).toEqual(expectedDate.toDateString());
+			}
+		});
+
 		test('.endOfMonth() should work correctly on a date', () => {
 			expect(evaluate('={{ DateTime.local(2023, 1, 16).endOfMonth() }}')).toEqual(
 				DateTime.local(2023, 1, 31, 23, 59, 59, 999, { zone: TEST_TIMEZONE }).toJSDate(),
@@ -30,6 +57,11 @@ describe('Data Transformation Functions', () => {
 			expect(evaluate('={{ DateTime.local(2023, 1, 20).extract("day") }}')).toEqual(20);
 		});

+		test('.extract() should extract week for no args', () => {
+			expect(evaluate('={{ DateTime.local(2023, 1, 20).extract() }}')).toEqual(3);
+		});
+
+
 		test('.format("yyyy LLL dd") should work correctly on a date', () => {
 			expect(evaluate('={{ DateTime.local(2023, 1, 16).format("yyyy LLL dd") }}')).toEqual(
 				'2023 Jan 16',
@@ -40,5 +72,29 @@ describe('Data Transformation Functions', () => {
 			const date = new Date(2022, 0, 3);
 			expect(evaluate(`={{ "${getLocalISOString(date)}".toDate() }}`)).toEqual(date);
 		});
+
+		test('.inBetween() should work on string and Date', () => {
+			expect(evaluate(`={{ $now.isBetween('2023-06-23'.toDate(), '2023-06-23') }}`)).toBeDefined();
+		});
+
+		test('.inBetween() should work on string and DateTime', () => {
+			expect(evaluate(`={{ $now.isBetween($now, '2023-06-23') }}`)).toBeDefined();
+		});
+
+		test('.inBetween() should not work for invalid strings', () => {
+			expect(evaluate(`={{ $now.isBetween($now, 'invalid') }}`)).toBeUndefined();
+		});
+
+		test('.inBetween() should not work for numbers', () => {
+			expect(evaluate(`={{ $now.isBetween($now, 1) }}`)).toBeUndefined();
+		});
+
+		test('.inBetween() should not work for a single argument', () => {
+			expect(() => evaluate(`={{ $now.isBetween($now) }}`)).toThrow();
+		});
+
+		test('.inBetween() should not work for a more than two arguments', () => {
+			expect(() => evaluate(`={{ $now.isBetween($now, '2023-06-23', '2023-09-21'.toDate()) }}`)).toThrow();
+		});
 	});
 });
--- a/packages/workflow/test/ExpressionExtensions/Helpers.ts
+++ b/packages/workflow/test/ExpressionExtensions/Helpers.ts
@@ -18,6 +18,9 @@ export const workflow = new Workflow({
 	connections: {},
 	active: false,
 	nodeTypes,
+	settings: {
+		timezone: TEST_TIMEZONE,
+	}
 });
 export const expression = new Expression(workflow);

--- a/packages/workflow/test/ExpressionExtensions/NumberExtensions.test.ts
+++ b/packages/workflow/test/ExpressionExtensions/NumberExtensions.test.ts
@@ -38,14 +38,22 @@ describe('Data Transformation Functions', () => {
 			expect(evaluate('={{ (9).isOdd() }}')).toEqual(true);
 			expect(evaluate('={{ (8).isOdd() }}')).toEqual(false);
 			expect(evaluate('={{ (0).isOdd() }}')).toEqual(false);
-			expect(evaluate('={{ (NaN).isOdd() }}')).toEqual(false);
+		});
+
+		test('.isOdd() should not work on a float or NaN', () => {
+			expect(() => evaluate('={{ (NaN).isOdd() }}')).toThrow();
+			expect(() => evaluate('={{ (9.2).isOdd() }}')).toThrow();
 		});

 		test('.isEven() should work on a number', () => {
 			expect(evaluate('={{ (9).isEven() }}')).toEqual(false);
 			expect(evaluate('={{ (8).isEven() }}')).toEqual(true);
 			expect(evaluate('={{ (0).isEven() }}')).toEqual(true);
-			expect(evaluate('={{ (NaN).isEven() }}')).toEqual(false);
+		});
+
+		test('.isEven() should not work on a float or NaN', () => {
+			expect(() => evaluate('={{ (NaN).isEven() }}')).toThrow();
+			expect(() => evaluate('={{ (9.2).isEven() }}')).toThrow();
 		});
 	});

--- a/packages/workflow/test/ExpressionExtensions/ObjectExtensions.test.ts
+++ b/packages/workflow/test/ExpressionExtensions/ObjectExtensions.test.ts
@@ -7,14 +7,6 @@ describe('Data Transformation Functions', () => {
 			expect(evaluate('={{({ test1: 1 }).isEmpty()}}')).toEqual(false);
 		});

-		test('.merge should work on an object', () => {
-			expect(evaluate('={{ ({ test1: 1, test2: 2 }).merge({ test2: 3, test3: 3 }) }}')).toEqual({
-				test1: 1,
-				test2: 2,
-				test3: 3,
-			});
-		});
-
 		test('.hasField should work on an object', () => {
 			expect(evaluate('={{ ({ test1: 1 }).hasField("test1") }}')).toEqual(true);
 			expect(evaluate('={{ ({ test1: 1 }).hasField("test2") }}')).toEqual(false);
@@ -44,6 +36,14 @@ describe('Data Transformation Functions', () => {
 			});
 		});

+		test('.removeFieldsContaining should not work for empty string', () => {
+			expect(
+				() => evaluate(
+					'={{ ({ test1: "i exist", test2: "i should be removed", test3: "i should also be removed" }).removeFieldsContaining("") }}',
+				),
+			).toThrow();
+		});
+
 		test('.keepFieldsContaining should work on an object', () => {
 			expect(
 				evaluate(
@@ -54,6 +54,24 @@ describe('Data Transformation Functions', () => {
 			});
 		});

+		test('.keepFieldsContaining should work on a nested object', () => {
+			expect(
+				evaluate(
+					'={{ ({ test1: "i exist", test2: "i should be removed", test3: { test4: "me too" } }).keepFieldsContaining("exist") }}',
+				),
+			).toEqual({
+				test1: 'i exist',
+			});
+		});
+
+		test('.keepFieldsContaining should not work for empty string', () => {
+			expect(
+				() => evaluate(
+					'={{ ({ test1: "i exist", test2: "i should be removed", test3: "i should also be removed" }).keepFieldsContaining("") }}',
+				),
+			).toThrow();
+		});
+
 		test('.compact should work on an object', () => {
 			expect(
 				evaluate('={{ ({ test1: 1, test2: "2", test3: undefined, test4: null }).compact() }}'),