feat(editor): Implement Resource Mapper component (#6207)

*  scaffolding
*  finished scaffolding
*  renamed types
*  updated subtitle
*  renamed functions file, UI updates
*  query parameters fixes, ui updates, refactoring
*  fixes for credentials test, setup for error parsing
*  rlc for schema and table, error handling tweaks
*  delete operation, new options
*  columns loader
*  linter fixes
*  where clauses setup
*  logic for processing where clauses
*  select operation
*  refactoring
*  data mode for insert and update, wip
*  data mapping, insert update, skip on conflict option
*  select columns with spaces fix
*  update operation update, wip
*  finished update operation
*  upsert operation
*  ui fixes
* Copy updates.
* Copy updates.
*  option to convert empty strings to nulls, schema checks
*  UI requested updates
*  ssh setup WIP
*  fixes, ssh WIP
*  ssh fixes, credentials
*  credentials testing update
*  uncaught error fix
*  clean up
*  address in use fix
*  improved error message
*  tests setup
*  unit tests wip
*  config files clean up
*  utils unit tests
*  refactoring
*  setup for testing operations, tests for deleteTable operation
*  executeQuery and insert operations tests
*  select, update, upsert operations tests
*  runQueries tests setup
*  hint to query
* Copy updates.
*  ui fixes
*  clean up
*  error message update
*  ui update
* Minor tweaks to query params decription.
* feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752)
*  Added initial resource mapping support in google sheets node
*  Wired mapping API endpoint with node-specific logic for fetching mapping fields
*  Implementing mapping fields logic for google sheets
*  Updating Google Sheets execute methods to support resource mapper fields
* 🚧 Added initial version of `ResourceLocator` component
* 👌 Added `update` mode to resource mapper modes
* 👌 Addressing PR feedback
* 👌 Removing leftover const reference
* 👕 Fixing lint errors
*  singlton for conections
*  credentials test fix, clean up
* feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814)
*  scaffolding
*  finished scaffolding
*  renamed types
*  updated subtitle
*  renamed functions file, UI updates
*  query parameters fixes, ui updates, refactoring
*  fixes for credentials test, setup for error parsing
*  rlc for schema and table, error handling tweaks
*  delete operation, new options
*  columns loader
*  linter fixes
*  where clauses setup
*  logic for processing where clauses
*  select operation
*  refactoring
*  data mode for insert and update, wip
*  data mapping, insert update, skip on conflict option
*  select columns with spaces fix
*  update operation update, wip
*  finished update operation
*  upsert operation
*  ui fixes
* Copy updates.
* Copy updates.
*  option to convert empty strings to nulls, schema checks
*  UI requested updates
*  ssh setup WIP
*  fixes, ssh WIP
*  ssh fixes, credentials
*  credentials testing update
*  uncaught error fix
*  clean up
*  address in use fix
*  improved error message
*  tests setup
*  unit tests wip
*  config files clean up
*  utils unit tests
*  refactoring
*  setup for testing operations, tests for deleteTable operation
*  executeQuery and insert operations tests
*  select, update, upsert operations tests
*  runQueries tests setup
*  hint to query
* Copy updates.
*  ui fixes
*  clean up
*  error message update
*  ui update
* Minor tweaks to query params decription.
*  Updated Postgres node to use resource mapper component
*  Implemented postgres <-> resource mapper type mapping
*  Updated Postgres node execution to use resource mapper fields in v3
* 🔥 Removing unused import
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>

* feat(core): Resource editor componend P0 (#5970)
*  Added inital value of mapping mode dropdown
*  Finished mapping mode selector
*  Finished implementing mapping mode selector
*  Implemented 'Columns to match on' dropdown
*  Implemented `loadOptionsDependOn` support in resource mapper
*  Implemented initial version of mapping fields
*  Implementing dependant fields watcher in new component setup
*  Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one
*  Handling matching columns correctly in UI
*  Saving and loading resourceMapper values in component
*  Implemented proper data saving and loading
*  ResourceMapper component refactor, fixing value save/load
*  Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key
*  Updated Google Sheets node to work with the new UI
*  Updating Postgres Node to work with new UI
*  Additional loading indicator that shown if there is no mapping mode selector
*  Removing hard-coded values, fixing matching columns ordering, refactoring
*  Updating field names in nodes
*  Fixing minor UI issues
*  Implemented matching fields filter logic
*  Moving loading label outside of fields list
*  Added initial unit tests for resource mapper
*  Finished default rendering test
*  Test refactoring
*  Finished unit tests
* 🔨 Updating the way i18n is used in resource mapper components
* ✔️ Fixing value to match on logic for postgres node
*  Hiding mapping fields when auto-map mode is selected
*  Syncing selected mapping mode between components
*  Fixing dateTime input rendering and adding update check to Postgres node
*  Properly handling database connections. Sending null for empty string values.
* 💄 Updated wording in the error message for non-existing rows
*  Fixing issues with selected matching values
* ✔️ Updating unit tests after matching logic update
*  Updating matching columns when new fields are loaded
*  Defaulting to null for empty parameter values
*  Allowing zero as valid value for number imputs
*  Updated list of types that use datepicker as widger
*  Using text inputs for time types
*  Initial mapping field rework
*  Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors
*  Added tooltip for columns that cannot be deleted
*  Saving deleted values in parameter value
*  Implemented control to add/remove mapping fields
*  Syncing field list with add field dropdown when changing dependent values
*  Not showing removed fields in matching columns selector. Updating wording in matching columns selector description
*  Implementing disabled states for add/remove all fields options
*  Saving removed columns separately, updating copy
*  Implemented resource mapper values validation
*  Updated validation logic and error input styling
*  Validating resource mapper fields when new nodes are added
*  Using node field words in validation, refactoring resource mapper component
*  Implemented schema syncing and add/remove all fields
*  Implemented custom parameter actions
*  Implemented loading indicator in parameter options
* 🔨 Removing unnecessary constants and vue props
*  Handling default values properly
*  Fixing validation logic
* 👕 Fixing lint errors
*  Fixing type issues
*  Not showing fields by default if `addAllFields` is set to `false`
*  Implemented field type validation in resource mapper
*  Updated casing in copy, removed all/remove all option from bottom menu
*  Added auto mapping mode notice
*  Added support for more types in validation
*  Added support for enumerated values
*  Fixing imports after merging
*  Not showing removed fields in matching columns selector. Refactoring validation logic.
* 👕 Fixing imports
* ✔️ Updating unit tests
*  Added resource mapper schema tests
*  Removing `match` from resource mapper field definition, fixing matching columns loading
*  Fixed schema merging
*  update operation return data fix
*  review
* 🐛 Added missing import
* 💄 Updating parameter actions icon based on the ui review
* 💄 Updating word capitalisation in tooltips
* 💄 Added empty state to mapping fields list
* 💄 Removing asterisk from fields, updating tooltips for matching fields
*  Preventing matching fields from being removed by 'Remove All option'
*  Not showing hidden fields in the `Add field` dropdown
*  Added support for custom matching columns labels
*  query optimization
*  fix
*  Optimizing Postgres node enumeration logic
*  Added empty state for matching columns
*  Only fully loading fields if there is no schema fetched
*  Hiding mapping fields if there is no matching columns available in the schema
* ✔️ Fixing minor issues
*  Implemented runtime type validation
* 🔨 Refactoring validation logic
*  Implemented required check, added more custom messages
*  Skipping boolean type in required check
* Type check improvements
*  Only reloading fields if dependent values actually change
*  Adding item index to validation error title
*  Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted
*  Resetting field values when adding them via the addAll option
*  Using minor version (2.2) for new Postgres node
*  Implemented proper date validation and type casting
* 👕 Consolidating typing
*  Added unit tests for type validations
* 👌 Addressing front-end review comments
*  More refactoring to address review changes
*  Updating leftover props
*  Added fallback for ISO dates with invalid timezones
* Added timestamp to datetime test cases
*  Reseting matching columns if operation changes
*  Not forcing auto-increment fields to be filled in in Postgres node. Handling null values
* 💄 Added a custom message for invalid dates
*  Better handling of JSON values
*  Updating codemirror readonly stauts based on component property, handling objects in json validation
* Deleting leftover console.log
*  Better time validation
*  Fixing build error after merging
* 👕 Fixing lint error
*  Updating node configuration values
*  Handling postgres arrays better
*  Handling SQL array syntax
*  Updating time validation rules to include timezone
*  Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets
*  Allowing removed fields to be selected for match
*  Updated the query for fetching unique columns and primary keys
*  Optimizing the unique query
*  Setting timezone to all parsed dates
*  Addressing PR review feedback
*  Configuring Sheets node for production, minor vue component update
* New cases added to the TypeValidation test.
*  Tweaking validation rules for arrays/objects and updating test cases
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
This commit is contained in:
Milorad FIlipović
2023-05-31 11:56:09 +02:00
committed by GitHub
parent 5ae1124106
commit 04cfa548af
57 changed files with 3436 additions and 183 deletions

View File

@@ -0,0 +1,392 @@
<script setup lang="ts">
import type { IUpdateInformation } from '@/Interface';
import type {
FieldType,
INodeIssues,
INodeParameters,
INodeProperties,
NodePropertyTypes,
ResourceMapperField,
ResourceMapperValue,
} from 'n8n-workflow';
import ParameterInputFull from '@/components/ParameterInputFull.vue';
import ParameterIssues from '../ParameterIssues.vue';
import ParameterOptions from '../ParameterOptions.vue';
import { computed } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNDVStore } from '@/stores';
import { fieldCannotBeDeleted, isMatchingField, parseResourceMapperFieldName } from '@/utils';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
parameter: INodeProperties;
path: string;
nodeValues: INodeParameters | undefined;
fieldsToMap: ResourceMapperField[];
paramValue: ResourceMapperValue;
labelSize: string;
showMatchingColumnsSelector: boolean;
showMappingModeSelect: boolean;
loading: boolean;
refreshInProgress: boolean;
}
const props = defineProps<Props>();
const FORCE_TEXT_INPUT_FOR_TYPES: FieldType[] = ['time', 'object', 'array'];
const {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
} = useNodeSpecificationValues(props.parameter.typeOptions);
const emit = defineEmits<{
(event: 'fieldValueChanged', value: IUpdateInformation): void;
(event: 'removeField', field: string): void;
(event: 'addField', field: string): void;
(event: 'refreshFieldList'): void;
}>();
const ndvStore = useNDVStore();
const fieldsUi = computed<INodeProperties[]>(() => {
return props.fieldsToMap
.filter((field) => field.display !== false && field.removed !== true)
.map((field) => {
return {
displayName: getFieldLabel(field),
// Set part of the path to each param name so value can be fetched properly by input parameter list component
name: `value["${field.id}"]`,
type: getParamType(field),
default: field.type === 'boolean' ? false : '',
required: field.required,
description: getFieldDescription(field),
options: field.options,
};
});
});
const orderedFields = computed<INodeProperties[]>(() => {
// Sort so that matching columns are first
if (props.paramValue.matchingColumns) {
fieldsUi.value.forEach((field, i) => {
const fieldName = parseResourceMapperFieldName(field.name);
if (fieldName) {
if (props.paramValue.matchingColumns.includes(fieldName)) {
fieldsUi.value.splice(i, 1);
fieldsUi.value.unshift(field);
}
}
});
}
return fieldsUi.value;
});
const removedFields = computed<ResourceMapperField[]>(() => {
return props.fieldsToMap.filter((field) => field.removed === true && field.display !== false);
});
const addFieldOptions = computed<Array<{ name: string; value: string; disabled?: boolean }>>(() => {
return removedFields.value.map((field) => {
return {
name: field.displayName,
value: field.id,
disabled: false,
};
});
});
const parameterActions = computed<Array<{ label: string; value: string; disabled?: boolean }>>(
() => {
return [
{
label: locale.baseText('resourceMapper.refreshFieldList', {
interpolate: { fieldWord: singularFieldWordCapitalized.value },
}),
value: 'refreshFieldList',
},
{
label: locale.baseText('resourceMapper.addAllFields', {
interpolate: { fieldWord: pluralFieldWordCapitalized.value },
}),
value: 'addAllFields',
disabled: removedFields.value.length === 0,
},
{
label: locale.baseText('resourceMapper.removeAllFields', {
interpolate: { fieldWord: pluralFieldWordCapitalized.value },
}),
value: 'removeAllFields',
disabled: isRemoveAllAvailable.value === false,
},
];
},
);
const isRemoveAllAvailable = computed<boolean>(() => {
return (
removedFields.value.length !== props.fieldsToMap.length &&
props.fieldsToMap.some((field) => {
return (
field.removed !== true &&
!fieldCannotBeDeleted(
field,
props.showMatchingColumnsSelector,
resourceMapperMode.value,
props.paramValue.matchingColumns,
)
);
})
);
});
const resourceMapperMode = computed<string | undefined>(() => {
return resourceMapperTypeOptions.value?.mode;
});
const valuesLabel = computed<string>(() => {
if (resourceMapperMode.value && resourceMapperMode.value === 'update') {
return locale.baseText('resourceMapper.valuesToUpdate.label');
}
return locale.baseText('resourceMapper.valuesToSend.label');
});
const fetchingFieldsLabel = computed<string>(() => {
return locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord.value,
},
});
});
function getFieldLabel(field: ResourceMapperField): string {
if (
isMatchingField(field.id, props.paramValue.matchingColumns, props.showMatchingColumnsSelector)
) {
const suffix = locale.baseText('resourceMapper.usingToMatch') || '';
return `${field.displayName} ${suffix}`;
}
return field.displayName;
}
function getFieldDescription(field: ResourceMapperField): string {
if (
isMatchingField(field.id, props.paramValue.matchingColumns, props.showMatchingColumnsSelector)
) {
return (
locale.baseText('resourceMapper.usingToMatch.description', {
interpolate: {
fieldWord: singularFieldWord.value,
},
}) || ''
);
}
return '';
}
function getParameterValue(parameterName: string) {
const fieldName = parseResourceMapperFieldName(parameterName);
if (fieldName && props.paramValue.value) {
return props.paramValue.value[fieldName];
}
return null;
}
function getFieldIssues(field: INodeProperties): string[] {
if (!ndvStore.activeNode) return [];
const nodeIssues = ndvStore.activeNode.issues || ({} as INodeIssues);
const fieldName = parseResourceMapperFieldName(field.name);
if (!fieldName) return [];
let fieldIssues: string[] = [];
const key = `${props.parameter.name}.${fieldName}`;
if (nodeIssues['parameters'] && key in nodeIssues['parameters']) {
fieldIssues = fieldIssues.concat(nodeIssues['parameters'][key]);
}
return fieldIssues;
}
function getParamType(field: ResourceMapperField): NodePropertyTypes {
if (field.type && !FORCE_TEXT_INPUT_FOR_TYPES.includes(field.type)) {
return field.type as NodePropertyTypes;
}
return 'string';
}
function onValueChanged(value: IUpdateInformation): void {
emit('fieldValueChanged', value);
}
function removeField(fieldName: string) {
emit('removeField', fieldName);
}
function addField(fieldName: string) {
emit('addField', fieldName);
}
function onParameterActionSelected(action: string): void {
switch (action) {
case 'addAllFields':
emit('addField', action);
break;
case 'removeAllFields':
emit('removeField', action);
break;
case 'refreshFieldList':
emit('refreshFieldList');
break;
default:
break;
}
}
defineExpose({
orderedFields,
});
</script>
<template>
<div class="mt-xs" data-test-id="mapping-fields-container">
<n8n-input-label
:label="valuesLabel"
:underline="true"
:size="labelSize"
:showOptions="true"
:showExpressionSelector="false"
color="text-dark"
>
<template #options>
<parameter-options
:parameter="parameter"
:customActions="parameterActions"
:loading="props.refreshInProgress"
:loadingMessage="fetchingFieldsLabel"
@optionSelected="onParameterActionSelected"
/>
</template>
</n8n-input-label>
<div v-if="orderedFields.length === 0" class="mt-3xs mb-xs">
<n8n-text size="small">{{
$locale.baseText('fixedCollectionParameter.currentlyNoItemsExist')
}}</n8n-text>
</div>
<div
v-for="field in orderedFields"
:key="field.name"
:class="{
['parameter-item']: true,
[$style.parameterItem]: true,
[$style.hasIssues]: getFieldIssues(field).length > 0,
}"
>
<div
v-if="resourceMapperMode === 'add' && field.required"
:class="['delete-option', 'mt-5xs', $style.parameterTooltipIcon]"
>
<n8n-tooltip placement="top">
<template #content>
<span>{{
locale.baseText('resourceMapper.mandatoryField.title', {
interpolate: { fieldWord: singularFieldWord },
})
}}</span>
</template>
<font-awesome-icon icon="question-circle" />
</n8n-tooltip>
</div>
<div
v-else-if="
!isMatchingField(
field.name,
props.paramValue.matchingColumns,
props.showMatchingColumnsSelector,
)
"
:class="['delete-option', 'clickable', 'mt-5xs']"
>
<font-awesome-icon
icon="trash"
:title="
locale.baseText('resourceMapper.removeField', {
interpolate: {
fieldWord: singularFieldWordCapitalized,
},
})
"
data-test-id="remove-field-button"
@click="removeField(field.name)"
/>
</div>
<div :class="$style.parameterInput">
<parameter-input-full
:parameter="field"
:value="getParameterValue(field.name)"
:displayOptions="true"
:path="`${props.path}.${field.name}`"
:isReadOnly="refreshInProgress"
:hideIssues="true"
:nodeValues="nodeValues"
:class="$style.parameterInputFull"
@valueChanged="onValueChanged"
/>
</div>
<parameter-issues
v-if="getFieldIssues(field).length > 0"
:issues="getFieldIssues(field)"
:class="[$style.parameterIssues, 'ml-5xs']"
/>
</div>
<div class="add-option" data-test-id="add-fields-select">
<n8n-select
:placeholder="
locale.baseText('resourceMapper.addFieldToSend', {
interpolate: { fieldWord: singularFieldWordCapitalized },
})
"
size="small"
:disabled="addFieldOptions.length == 0"
@change="addField"
>
<n8n-option
v-for="item in addFieldOptions"
:key="item.value"
:label="item.name"
:value="item.value"
:disabled="item.disabled"
>
</n8n-option>
</n8n-select>
</div>
</div>
</template>
<style module lang="scss">
.parameterItem {
display: flex;
padding: 0 0 0 1em;
.parameterInput {
width: 100%;
}
&.hasIssues {
.parameterIssues {
float: none;
padding-top: var(--spacing-xl);
}
input,
input:focus {
--input-border-color: var(--color-danger);
border-color: var(--color-danger);
}
}
}
.parameterTooltipIcon {
color: var(--color-text-light) !important;
}
</style>