Commit Graph

700 Commits

Author SHA1 Message Date
Iván Ovejero
b4ebb1a28d fix(core): Account for non-ASCII chars in filename on binary data download (#7742)
https://n8nio.sentry.io/issues/4641538638
2023-11-17 10:07:44 +01:00
Ricardo Espinoza
8e0ae3cf8c refactor: Extract Invitation routes to InvitationController (no-changelog) (#7726)
This PR:

- Creates `InvitationController`
- Moves `POST /users` to `POST /invitations` and move related test to
`invitations.api.tests`
- Moves `POST /users/:id` to `POST /invitations/:id/accept` and move
related test to `invitations.api.tests`
- Adjusts FE to use new endpoints
- Moves all the invitation logic to the `UserService`

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-16 12:39:43 -05:00
Val
d39bb2540f feat: Add scopes to /login endpoint (no-changelog) (#7718)
Github issue / Community forum post (link here to close automatically):
2023-11-16 11:11:55 +00:00
Ricardo Espinoza
4020c14d59 refactor: Use POST /users to re-invite users (no-changelog) (#7714) 2023-11-15 06:40:57 -05:00
कारतोफ्फेलस्क्रिप्ट™
3460eb5eeb fix(core): Initialize JWT Secret before it's used anywhere (#7707)
HELP-394
2023-11-15 12:17:18 +01:00
Iván Ovejero
b2ca050031 perf(core): Lazyload security audit reporters (#7696)
Also converting to service.

Followup to https://github.com/n8n-io/n8n/pull/7663
2023-11-13 11:50:43 +01:00
Ricardo Espinoza
0ddafd2b82 test: Unify users.controller.test and users.api.test (no-changelog) (#7658)
Groundwork to be able to safely refactor and move the invitation logic
to the UserService.

Fixes ADO-1358

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-10 09:41:58 -05:00
कारतोफ्फेलस्क्रिप्ट™
000e76e3b4 ci(core): Reduce memory usage in tests (part-2) (no-changelog) (#7671)
This also gets rid of `Db.collection`, which was another source of
circular dependencies.
2023-11-10 15:04:26 +01:00
कारतोफ्फेलस्क्रिप्ट™
5edf722209 ci: Stop disabling linting rules that are explicitly set to warn (no-changelog) (#7669) 2023-11-09 14:05:39 +01:00
कारतोफ्फेलस्क्रिप्ट™
0346b211a7 ci(core): Reduce memory usage in tests (part-1) (no-changelog) (#7654) 2023-11-08 16:29:39 +01:00
Iván Ovejero
48c068f97b fix(core): Comply with custom default for workflow saving settings (#7634)
https://linear.app/n8n/issue/PAY-982
2023-11-07 16:26:55 +01:00
Tomi Turtiainen
ac877014ed feat(core): Initial support for two-way communication over websockets (#7570)
- Enable two-way communication with web sockets
- Enable sending push messages to specific users
- Add collaboration service for managing active users for workflow

Missing things:
- State is currently kept only in memory, making this not work in
multi-master setups
- Removing a user from active users in situations where they go inactive
or we miss the "workflow closed" message
- I think a timer based solution for this would cover most edge cases.
I.e. have FE ping every X minutes, BE removes the user unless they have
received a ping in Y minutes, where Y > X
- FE changes to be added later by @MiloradFilipovic 

Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-07 17:26:45 +02:00
कारतोफ्फेलस्क्रिप्ट™
f0fc5b16d3 ci(core): Load config schema after process.env has been overwritten (no-changelog) (#7550) 2023-11-07 15:58:28 +01:00
कारतोफ्फेलस्क्रिप्ट™
60314248f4 fix(core): Make password-reset urls valid only for single-use (#7622) 2023-11-07 15:35:43 +01:00
Iván Ovejero
c857e42677 feat(core): Coordinate workflow activation in multiple main scenario in internal API (#7566)
Story: https://linear.app/n8n/issue/PAY-926

This PR coordinates workflow activation on instance startup and on
leadership change in multiple main scenario in the internal API. Part 3
on manual workflow activation and deactivation will be a separate PR.

### Part 1: Instance startup

In multi-main scenario, on starting an instance...
- [x] If the instance is the leader, it should add webhooks, triggers
and pollers.
- [x] If the instance is the follower, it should not add webhooks,
triggers or pollers.
- [x] Unit tests.

### Part 2: Leadership change 

In multi-main scenario, if the main instance leader dies…

- [x] The new main instance leader must activate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] The old main instance leader must deactivate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] Unit tests.

To test, start two instances and check behavior on startup and
leadership change:

```
EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug npm run start

EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug N8N_PORT=5679 npm run start
```
2023-11-07 13:48:48 +01:00
कारतोफ्फेलस्क्रिप्ट™
acec9bad71 refactor(core): Convert OAuth1/OAuth2 routes to decorated controller classes (no-changelog) (#5973) 2023-11-03 17:20:54 +01:00
Iván Ovejero
1c77d6597f refactor(core): Ensure only leader handles licensing in multi-main scenario (#7558)
https://linear.app/n8n/issue/PAY-953/ensure-only-main-instance-leader-handles-licensing
2023-11-02 14:16:22 +01:00
Iván Ovejero
921d213ae5 refactor(core): Create pruning service (no-changelog) (#7564)
https://linear.app/n8n/issue/PAY-954/ensure-only-main-instance-leader-handles-pruning
2023-11-02 12:24:25 +01:00
Jon
479f90231d fix(core): Fix issue that prevents owner logging in when using ldap (#7408)
This PR prioritises the internal email account over LDAP for the Owner.

---------

Co-authored-by: ricardo <ricardoespinoza105@gmail.com>
2023-11-01 23:02:49 -04:00
Omar Ajoue
437c95e84e fix(core): Permission check for subworkflow properly checking for workflow settings (#7576)
The `sharing` related code is legacy that was not removed. Subworkflow
execution should check workflow settings alone, and this is now
reflected in the code.

Github issue / Community forum post (link here to close automatically):

https://community.n8n.io/t/bug-when-using-the-execute-workflow-node-when-workflow-is-shared/32207

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-11-01 18:31:34 +01:00
Michael Auerswald
b350568505 fix(core): Fix data decryption on credentials import (#7560)
Due to a change, during the credentials import command, the core's
Credential object is being called through its prototype. This caused the
Credential's cipher variable to not be set, thus no cipher service being
available during import. This fix catches this edge case and provides a
fix.
2023-10-31 13:15:09 +01:00
Iván Ovejero
442c73e63b feat(core): Set up leader selection for multiple main instances (#7527)
https://linear.app/n8n/issue/PAY-933/set-up-leader-selection-for-multiple-main-instances

- [x] Set up new envs
- [x] Add config and license checks
- [x] Implement `MultiMainInstancePublisher`
- [x] Expand `RedisServicePubSubPublisher` to support
`MultiMainInstancePublisher`
- [x] Init `MultiMainInstancePublisher` on startup and destroy on
shutdown
- [x] Add to sandbox plans
- [x] Test manually

Note: This is only for setup - coordinating in reaction to leadership
changes will come in later PRs.
2023-10-30 16:22:32 +01:00
कारतोफ्फेलस्क्रिप्ट™
35bb42c1b9 refactor(core): Avoid passing around static state like default timezone (no-changelog) (#7221) 2023-10-27 14:17:52 +02:00
Iván Ovejero
62c096710f refactor: Run lintfix (no-changelog) (#7537)
- Fix autofixable violations
- Remove unused directives
- Allow for PascalCased variables - needed for dynamically imported or
assigned classes, decorators, routers, etc.
2023-10-27 14:15:02 +02:00
Iván Ovejero
ae8c7a635e refactor(core)!: Remove webhook deregistration at startup and shutdown (#7515)
https://linear.app/n8n/issue/PAY-932/deprecate-flag-to-skip-webhook-deregistration-on-shutdown
2023-10-26 14:37:54 +02:00
Iván Ovejero
5477e3fb45 refactor(core): Move execution save settings into lifecycle function (no-changelog) (#7370)
Move the handling of execution save settings into a tested lifecycle
function as discussed with Omar
2023-10-26 14:35:38 +02:00
कारतोफ्फेलस्क्रिप्ट™
05586a900d refactor(core): Make Logger a service (no-changelog) (#7494) 2023-10-25 16:35:22 +02:00
Val
93cfabbeac fix(core): Fix workflow activation with history and workflow history for EE (no-changelog) (#7508)
Github issue / Community forum post (link here to close automatically):
2023-10-25 11:07:11 +01:00
Iván Ovejero
671c95760b refactor(core): Make executions pruning more resilient (#7480)
This PR converts the hard-deletion interval to a timeout:
- to prevent the interval from not being restored when hard deletion
throws, and
- to prevent a long-running hard deletion from leading to duplicate
deletions.
2023-10-24 16:16:45 +02:00
Cornelius Suermann
3ef771ea38 feat: Collect usage metrics on license renewal (no-changelog) (#7486) 2023-10-23 21:39:29 +02:00
Val
41236b7e08 fix: Save new version of the workflow instead of the previous (no-changelog) (#7428)
Github issue / Community forum post (link here to close automatically):
2023-10-23 16:30:36 +02:00
कारतोफ्फेलस्क्रिप्ट™
b6de910cbe refactor(core): Abstract away InstanceSettings and encryptionKey into injectable services (no-changelog) (#7471)
This change ensures that things like `encryptionKey` and `instanceId`
are always available directly where they are needed, instead of passing
them around throughout the code.
2023-10-23 13:39:35 +02:00
Iván Ovejero
0b42d1aa71 refactor(core): Limit soft-deletions to pruning only (#7469)
Based on customer feedback, we should limit soft deletions to pruning
only, to prevent executions from piling up in very high volume cases.
2023-10-20 15:02:47 +02:00
कारतोफ्फेलस्क्रिप्ट™
55c6a1b0d3 fix(core): Do not return inviteAcceptUrl in response if email was sent (#7465) 2023-10-19 13:58:06 +02:00
कारतोफ्फेलस्क्रिप्ट™
83762e051d fix(core): Handle gzip and deflate compressed request payloads (#7461)
NODE-870
2023-10-18 21:52:15 +02:00
Michael Auerswald
e01b9e5ae1 fix(core): Prevent false stalled jobs in queue mode from displaying as errored (#7435)
This is related to an issue with how Bull handles stalled jobs, see
https://github.com/OptimalBits/bull/issues/1415 for reference.

CPU intensive workflows can in certain cases take a long while to finish
up, thereby blocking the thread and causing Bull queue to think the job
has stalled, even though it finished successfully. In these cases the
error handling could then overwrite the successful execution data with
the error message.
2023-10-13 16:07:08 +02:00
Omar Ajoue
46977a2aff fix: Prevent undefined issues when restoring binary data (#7419)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-10-12 16:41:51 +02:00
Michael Auerswald
2f4d91b2cd fix(core): Fix ignoring crashed executions without event msgs (#7368)
when the event logs do not contain messages for running executions, the
recovery/crash detection on startup would skip these. this PR fixes
that.
2023-10-11 14:22:43 +02:00
कारतोफ्फेलस्क्रिप्ट™
6479eb180f fix(Webhook Node): Backward compatible form-data parsing for non-array files (#7385)
Fixes
https://community.n8n.io/t/possible-bug-0-added-to-end-of-files-sent-via-webhook/31169
2023-10-10 15:19:05 +02:00
Iván Ovejero
0847623f85 feat(core): Switch binary filesystem mode to nested path structure (#7307)
Depends on #7253 | Story:
[PAY-863](https://linear.app/n8n/issue/PAY-863/switch-binary-filesystem-mode-to-nested-path-structure)

This PR introduces `filesystem-v2` to store binary data in the
filesystem in the same format as `s3`.
2023-10-10 10:06:06 +02:00
कारतोफ्फेलस्क्रिप्ट™
c5ee06cc61 refactor(core): Refactor nodes loading (no-changelog) (#7283)
fixes PAY-605
2023-10-09 16:09:23 +02:00
Iván Ovejero
34bda535e6 refactor(core): Create controller for binary data (no-changelog) (#7363)
This PR adds a controller for binary data + integration tests.
2023-10-06 16:21:13 +02:00
Michael Auerswald
afa683a06f refactor(core): Have one orchestration service per instance type (#7303)
webhook instances will not listen to either worker or event log messages
on the Redis pub/sub channel
2023-10-06 13:58:11 +02:00
Cornelius Suermann
f4d8c9eed5 feat: Improve error msg when attempting to redeem the same activation code multiple times (no-changelog) (#7355) 2023-10-05 17:08:30 +02:00
Iván Ovejero
1a661e6d00 feat(core): Integrate object store as binary data manager (#7253)
Depends on: #7225 | Story:
[PAY-848](https://linear.app/n8n/issue/PAY-848)

This PR integrates the object store service as a new binary data manager
for Enterprise.
2023-10-05 15:25:17 +02:00
कारतोफ्फेलस्क्रिप्ट™
169175080a ci: Fix WorkflowHistoryManager tests (no-changelog) (#7356)
[DB Tests](https://github.com/n8n-io/n8n/actions/runs/6418058186)
2023-10-05 13:14:57 +02:00
Tomi Turtiainen
1b4848afcb fix(core): Fix pruning of non-finished executions (#7333)
This fixes a bug in the pruning (soft-delete). The pruning was a bit too
aggressive, as it also pruned executions that weren't in an end state
yet. This only becomes an issue if there are long-running executions
(e.g. workflow with Wait node) or the prune parameters are set to keep
only a tiny number of executions.
2023-10-04 16:32:05 +03:00
Val
0adc533719 feat: Workflow History pruning and prune time settings (#7343)
Github issue / Community forum post (link here to close automatically):
2023-10-04 13:57:21 +01:00
Iván Ovejero
1691223789 fix(core): Account for itemless case on restoring binary data ID (#7305)
https://linear.app/n8n/issue/PAY-862
2023-10-02 09:13:55 +02:00
Omar Ajoue
ec0379378e feat(core): Add plan name to telemetry (no-changelog) (#7296)
Github issue / Community forum post (link here to close automatically):
2023-09-29 15:41:32 +02:00