mirror of
https://github.com/Abdulazizzn/n8n-enterprise-unlocked.git
synced 2025-12-17 18:12:04 +00:00
feat(editor): Implement Resource Mapper component (#6207)
* ⚡ scaffolding * ⚡ finished scaffolding * ⚡ renamed types * ⚡ updated subtitle * ⚡ renamed functions file, UI updates * ⚡ query parameters fixes, ui updates, refactoring * ⚡ fixes for credentials test, setup for error parsing * ⚡ rlc for schema and table, error handling tweaks * ⚡ delete operation, new options * ⚡ columns loader * ⚡ linter fixes * ⚡ where clauses setup * ⚡ logic for processing where clauses * ⚡ select operation * ⚡ refactoring * ⚡ data mode for insert and update, wip * ⚡ data mapping, insert update, skip on conflict option * ⚡ select columns with spaces fix * ⚡ update operation update, wip * ⚡ finished update operation * ⚡ upsert operation * ⚡ ui fixes * Copy updates. * Copy updates. * ⚡ option to convert empty strings to nulls, schema checks * ⚡ UI requested updates * ⚡ ssh setup WIP * ⚡ fixes, ssh WIP * ⚡ ssh fixes, credentials * ⚡ credentials testing update * ⚡ uncaught error fix * ⚡ clean up * ⚡ address in use fix * ⚡ improved error message * ⚡ tests setup * ⚡ unit tests wip * ⚡ config files clean up * ⚡ utils unit tests * ⚡ refactoring * ⚡ setup for testing operations, tests for deleteTable operation * ⚡ executeQuery and insert operations tests * ⚡ select, update, upsert operations tests * ⚡ runQueries tests setup * ⚡ hint to query * Copy updates. * ⚡ ui fixes * ⚡ clean up * ⚡ error message update * ⚡ ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * ⚡ singlton for conections * ⚡ credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * ⚡ scaffolding * ⚡ finished scaffolding * ⚡ renamed types * ⚡ updated subtitle * ⚡ renamed functions file, UI updates * ⚡ query parameters fixes, ui updates, refactoring * ⚡ fixes for credentials test, setup for error parsing * ⚡ rlc for schema and table, error handling tweaks * ⚡ delete operation, new options * ⚡ columns loader * ⚡ linter fixes * ⚡ where clauses setup * ⚡ logic for processing where clauses * ⚡ select operation * ⚡ refactoring * ⚡ data mode for insert and update, wip * ⚡ data mapping, insert update, skip on conflict option * ⚡ select columns with spaces fix * ⚡ update operation update, wip * ⚡ finished update operation * ⚡ upsert operation * ⚡ ui fixes * Copy updates. * Copy updates. * ⚡ option to convert empty strings to nulls, schema checks * ⚡ UI requested updates * ⚡ ssh setup WIP * ⚡ fixes, ssh WIP * ⚡ ssh fixes, credentials * ⚡ credentials testing update * ⚡ uncaught error fix * ⚡ clean up * ⚡ address in use fix * ⚡ improved error message * ⚡ tests setup * ⚡ unit tests wip * ⚡ config files clean up * ⚡ utils unit tests * ⚡ refactoring * ⚡ setup for testing operations, tests for deleteTable operation * ⚡ executeQuery and insert operations tests * ⚡ select, update, upsert operations tests * ⚡ runQueries tests setup * ⚡ hint to query * Copy updates. * ⚡ ui fixes * ⚡ clean up * ⚡ error message update * ⚡ ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * ⚡ update operation return data fix * ⚡ review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * ⚡ query optimization * ⚡ fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
This commit is contained in:
Milorad FIlipović
committed by
GitHub
GitHub
parent
5ae1124106
commit
04cfa548af
206
packages/workflow/test/TypeValidation.test.ts
Normal file
206
packages/workflow/test/TypeValidation.test.ts
Normal file
@@ -0,0 +1,206 @@
|
||||
import { validateFieldType } from '@/NodeHelpers';
|
||||
import type { DateTime } from 'luxon';
|
||||
|
||||
const VALID_ISO_DATES = [
|
||||
'1994-11-05T08:15:30-05:00',
|
||||
'1994-11-05T13:15:30Z',
|
||||
'1997-07-16T19:20+01:00',
|
||||
'1997-07-16T19:20:30+01:00',
|
||||
'1997-07-16T19:20:30.45+01:00',
|
||||
'2018-05-16',
|
||||
'1972-06-30T23:59:40Z',
|
||||
'2019-03-26T14:00:00.9Z',
|
||||
'2019-03-26T14:00:00.4999Z',
|
||||
'2023-05-17T10:52:32+0000',
|
||||
'2023-05-17T10:52:32+0000',
|
||||
];
|
||||
|
||||
const VALID_HTTP_DATES = [
|
||||
'Wed, 21 Oct 2015 07:28:00 GMT',
|
||||
'Wed, 01 Jun 2022 08:00:00 GMT',
|
||||
'Tue, 15 Nov 1994 12:45:26 GMT',
|
||||
'Wed, 1 Jun 2022 08:00:00 GMT',
|
||||
];
|
||||
|
||||
const VALID_RFC_DATES = [
|
||||
'Tue, 04 Jun 2013 07:40:03 -0400',
|
||||
'Tue, 4 Jun 2013 02:24:39 +0530',
|
||||
'Wed, 17 May 2023 10:52:32 +0000',
|
||||
];
|
||||
|
||||
const VALID_SQL_DATES = ['2008-11-11', '2008-11-11 13:23:44'];
|
||||
|
||||
const INVALID_DATES = [
|
||||
'1994-11-05M08:15:30-05:00',
|
||||
'18-05-2020',
|
||||
'',
|
||||
'Wed, 17 May 2023 10:52:32',
|
||||
'SMT, 17 May 2023 10:52:32',
|
||||
'1685084980', // We are not supporting timestamps
|
||||
'1685085012135',
|
||||
1685084980,
|
||||
1685085012135,
|
||||
true,
|
||||
[],
|
||||
];
|
||||
|
||||
describe('Type Validation', () => {
|
||||
it('should validate and cast ISO dates', () => {
|
||||
VALID_ISO_DATES.forEach((date) => {
|
||||
const validationResult = validateFieldType('date', date, 'dateTime');
|
||||
expect(validationResult.valid).toBe(true);
|
||||
expect((validationResult.newValue as DateTime).isValid).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it('should validate and cast RFC 2822 dates', () => {
|
||||
VALID_RFC_DATES.forEach((date) => {
|
||||
const validationResult = validateFieldType('date', date, 'dateTime');
|
||||
expect(validationResult.valid).toBe(true);
|
||||
expect((validationResult.newValue as DateTime).isValid).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it('should validate and cast HTTP dates', () => {
|
||||
VALID_HTTP_DATES.forEach((date) => {
|
||||
const validationResult = validateFieldType('date', date, 'dateTime');
|
||||
expect(validationResult.valid).toBe(true);
|
||||
expect((validationResult.newValue as DateTime).isValid).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it('should validate and cast SQL dates', () => {
|
||||
VALID_SQL_DATES.forEach((date) => {
|
||||
const validationResult = validateFieldType('date', date, 'dateTime');
|
||||
expect(validationResult.valid).toBe(true);
|
||||
expect((validationResult.newValue as DateTime).isValid).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it('should not validate invalid dates', () => {
|
||||
INVALID_DATES.forEach((date) => {
|
||||
const validationResult = validateFieldType('date', date, 'dateTime');
|
||||
expect(validationResult.valid).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
it('should validate boolean values properly', () => {
|
||||
expect(validateFieldType('boolean', 'true', 'boolean').newValue).toBe(true);
|
||||
expect(validateFieldType('boolean', 'TRUE', 'boolean').newValue).toBe(true);
|
||||
expect(validateFieldType('boolean', 1, 'boolean').newValue).toBe(true);
|
||||
expect(validateFieldType('boolean', '1', 'boolean').newValue).toBe(true);
|
||||
expect(validateFieldType('boolean', '01', 'boolean').newValue).toBe(true);
|
||||
expect(validateFieldType('boolean', 'false', 'boolean').newValue).toBe(false);
|
||||
expect(validateFieldType('boolean', 'FALSE', 'boolean').newValue).toBe(false);
|
||||
expect(validateFieldType('boolean', '0', 'boolean').newValue).toBe(false);
|
||||
expect(validateFieldType('boolean', '000', 'boolean').newValue).toBe(false);
|
||||
expect(validateFieldType('boolean', '0000', 'boolean').newValue).toBe(false);
|
||||
expect(validateFieldType('boolean', 0, 'boolean').newValue).toBe(false);
|
||||
});
|
||||
|
||||
it('should not validate invalid boolean values', () => {
|
||||
expect(validateFieldType('boolean', 'tru', 'boolean').valid).toBe(false);
|
||||
expect(validateFieldType('boolean', 'fals', 'boolean').valid).toBe(false);
|
||||
expect(validateFieldType('boolean', 1111, 'boolean').valid).toBe(false);
|
||||
expect(validateFieldType('boolean', 2, 'boolean').valid).toBe(false);
|
||||
expect(validateFieldType('boolean', -1, 'boolean').valid).toBe(false);
|
||||
expect(validateFieldType('boolean', 'yes', 'boolean').valid).toBe(false);
|
||||
expect(validateFieldType('boolean', 'no', 'boolean').valid).toBe(false);
|
||||
});
|
||||
|
||||
it('should validate and cast numbers', () => {
|
||||
expect(validateFieldType('number', '1', 'number').newValue).toBe(1);
|
||||
expect(validateFieldType('number', '-1', 'number').newValue).toBe(-1);
|
||||
expect(validateFieldType('number', '1.1', 'number').newValue).toBe(1.1);
|
||||
expect(validateFieldType('number', '-1.1', 'number').newValue).toBe(-1.1);
|
||||
expect(validateFieldType('number', 1, 'number').newValue).toBe(1);
|
||||
expect(validateFieldType('number', 'A', 'number').valid).toBe(false);
|
||||
expect(validateFieldType('number', '1,1', 'number').valid).toBe(false);
|
||||
expect(validateFieldType('number', true, 'number').valid).toBe(true);
|
||||
expect(validateFieldType('number', '1972-06-30T23:59:40Z', 'number').valid).toBe(false);
|
||||
expect(validateFieldType('number', [1, 2], 'number').valid).toBe(false);
|
||||
});
|
||||
|
||||
it('should validate and cast JSON properly', () => {
|
||||
expect(validateFieldType('json', '{"a": 1}', 'object').newValue).toEqual({ a: 1 });
|
||||
expect(
|
||||
validateFieldType('json', '{"a": 1, "b": { "c": 10, "d": "test"}}', 'object').valid,
|
||||
).toEqual(true);
|
||||
expect(validateFieldType('json', { name: 'John' }, 'object').valid).toEqual(true);
|
||||
expect(
|
||||
validateFieldType(
|
||||
'json',
|
||||
{ name: 'John', address: { street: 'Via Roma', city: 'Milano' } },
|
||||
'object',
|
||||
).valid,
|
||||
).toEqual(true);
|
||||
// Invalid value:
|
||||
expect(validateFieldType('json', ['one', 'two'], 'object').valid).toEqual(false);
|
||||
// eslint-disable-next-line prettier/prettier
|
||||
expect(validateFieldType('json', ["one", "two"], 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', '1', 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', '[1]', 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', '1.1', 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', 1.1, 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', '"a"', 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', '{a: 1}', 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', '["apples", "oranges"]', 'object').valid).toEqual(false);
|
||||
expect(validateFieldType('json', [{ name: 'john' }, { name: 'bob' }], 'object').valid).toEqual(
|
||||
false,
|
||||
);
|
||||
expect(
|
||||
validateFieldType('json', '[ { name: "john" }, { name: "bob" } ]', 'object').valid,
|
||||
).toEqual(false);
|
||||
});
|
||||
|
||||
it('should validate and cast arrays properly', () => {
|
||||
expect(validateFieldType('array', '["apples", "oranges"]', 'array').newValue).toEqual([
|
||||
'apples',
|
||||
'oranges',
|
||||
]);
|
||||
expect(validateFieldType('array', '[1]', 'array').newValue).toEqual([1]);
|
||||
expect(validateFieldType('array', '[1, 2]', 'array').newValue).toEqual([1, 2]);
|
||||
// Invalid values:
|
||||
expect(validateFieldType('array', '"apples", "oranges"', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '1', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '1.1', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '1, 2', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '1. 2. 3', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '[1, 2, 3', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '1, 2, 3]', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '{1, 2, {3, 4}, 5}', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', '1, 2, {3, 4}, 5', 'array').valid).toEqual(false);
|
||||
expect(validateFieldType('array', { name: 'John' }, 'array').valid).toEqual(false);
|
||||
});
|
||||
|
||||
it('should validate options properly', () => {
|
||||
expect(
|
||||
validateFieldType('options', 'oranges', 'options', [
|
||||
{ name: 'apples', value: 'apples' },
|
||||
{ name: 'oranges', value: 'oranges' },
|
||||
]).valid,
|
||||
).toEqual(true);
|
||||
expect(
|
||||
validateFieldType('options', 'something else', 'options', [
|
||||
{ name: 'apples', value: 'apples' },
|
||||
{ name: 'oranges', value: 'oranges' },
|
||||
]).valid,
|
||||
).toEqual(false);
|
||||
});
|
||||
|
||||
it('should validate and cast time properly', () => {
|
||||
expect(validateFieldType('time', '23:23', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '23:23:23', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '23:23:23+1000', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '23:23:23-1000', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '22:00:00+01:00', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '22:00:00-01:00', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '22:00:00+01', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '22:00:00-01', 'time').valid).toEqual(true);
|
||||
expect(validateFieldType('time', '23:23:23:23', 'time').valid).toEqual(false);
|
||||
expect(validateFieldType('time', '23', 'time').valid).toEqual(false);
|
||||
expect(validateFieldType('time', 'foo', 'time').valid).toEqual(false);
|
||||
expect(validateFieldType('time', '23:23:', 'time').valid).toEqual(false);
|
||||
expect(validateFieldType('time', '23::23::23', 'time').valid).toEqual(false);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user