Commit Graph

52 Commits

Author SHA1 Message Date
कारतोफ्फेलस्क्रिप्ट™
4b11268a6e refactor(core): Implement a new OnPubSubEvent decorator (#15688) 2025-06-04 18:32:33 +02:00
Iván Ovejero
3079059e96 refactor(core): Tear down OrchestrationService (#15100) 2025-05-09 09:45:27 +02:00
कारतोफ्फेलस्क्रिप्ट™
d80b49d6e5 feat(core): Setup backend modules (no-changelog) (#14084)
Co-authored-by: Guillaume Jacquart <jacquart.guillaume@gmail.com>
Co-authored-by: Danny Martini <danny@n8n.io>
2025-03-20 19:54:27 +01:00
कारतोफ्फेलस्क्रिप्ट™
e0fd50554d fix(core): Allow running webhook servers in multi-main mode (#13989) 2025-03-18 13:59:20 +01:00
Iván Ovejero
3ca99194c6 refactor(core): Migrate all errors in cli package to new hierarchy (#13478)
Co-authored-by: Tomi Turtiainen <10324676+tomi@users.noreply.github.com>
2025-02-27 09:30:55 +02:00
कारतोफ्फेलस्क्रिप्ट™
05b5f95331 refactor(core): Make external hooks type-safe, and add tests (#12893) 2025-01-29 10:33:39 +01:00
कारतोफ्फेलस्क्रिप्ट™
39d5e0ff87 refactor(core): Replace typedi with our custom DI system (no-changelog) (#12389)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2025-01-06 10:21:24 +01:00
Iván Ovejero
be50a9ac44 chore(core): Bring multi-main setup in line with scaling services (#11289) 2024-10-17 10:47:17 +02:00
Iván Ovejero
76ab780cdd refactor(core): Remove dead orchestration code (#11266) 2024-10-15 16:47:31 +02:00
Iván Ovejero
05467fd101 refactor(core): Move queueModeId as hostId to InstanceSettings (#11262) 2024-10-15 14:55:13 +02:00
Iván Ovejero
0820cb5ab9 refactor(core): Simplify main pubsub message handler (#11156) 2024-10-11 10:31:33 +02:00
Shireen Missi
52dd2c7619 feat(core): Dedupe (#10101)
Co-authored-by: Jan Oberhauser <jan@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Tomi Turtiainen <10324676+tomi@users.noreply.github.com>
Co-authored-by: Elias Meire <elias@meire.dev>
2024-10-10 16:12:05 +01:00
Iván Ovejero
383b4765d2 refactor(core): Simplify worker pubsub message handler (#11086) 2024-10-07 16:19:58 +02:00
Iván Ovejero
73a0d48073 refactor(core): Simplify webhook pubsub message handler (#11048) 2024-10-04 09:28:53 +02:00
कारतोफ्फेलस्क्रिप्ट™
25c8a328a8 refactor(core): Move instanceType to InstanceSettings (no-changelog) (#10640) 2024-09-16 13:37:14 +02:00
Tomi Turtiainen
5156313074 refactor(core): Enable import/order eslint rule (#10794) 2024-09-12 19:07:18 +03:00
Iván Ovejero
f667b384c9 refactor(core): Standardize filenames in cli (no-changelog) (#10484)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-08-22 11:10:37 +02:00
कारतोफ्फेलस्क्रिप्ट™
1869c396f3 fix(core): Do not load ScalingService in regular mode (no-changelog) (#10333) 2024-08-08 17:35:17 +02:00
Iván Ovejero
e31d017bdd refactor(core): Centralize scaling mode (no-changelog) (#9835) 2024-08-07 13:50:46 +02:00
Tomi Turtiainen
c8d322a9ba refactor(core): Reorganize webhook related components under src/webhooks (no-changelog) (#10296) 2024-08-07 11:23:44 +03:00
कारतोफ्फेलस्क्रिप्ट™
88086a41ff feat(core): Support community packages in scaling-mode (#10228) 2024-08-05 11:52:06 +02:00
कारतोफ्फेलस्क्रिप्ट™
c04f92f7fd refactor(core): Couple of refactors on WorkflowRunner and ActiveExecutions (no-changelog) (#8487) 2024-02-06 18:09:50 +01:00
Iván Ovejero
28aee7c425 refactor(core): Lock webhook process out of multi-main setup (no-changelog) (#8549) 2024-02-05 18:06:54 +01:00
कारतोफ्फेलस्क्रिप्ट™
913c8c6b0c feat(core): Upgrade oclif (no-changelog) (#8381) 2024-01-22 18:25:36 +01:00
Tomi Turtiainen
3a881be6c2 feat(core): Unify application components shutdown (#8097)
## Summary

Add `ShutdownService` and `OnShutdown` decorator for more unified way to
shutdown different components. Use this new way in the following
components:

- HTTP(S) server
- Pruning service
- Push connection
- License

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-22 12:39:58 +02:00
Tomi Turtiainen
4cae976a3b fix(core): Consider timeout in shutdown an error (#8050)
If the process doesn't shutdown within a time limit, exit with error
code.

1. conceptually something timing out is an error.
2. on successful exit we close down the DB connection gracefully. On an
exit timeout we rather not do that, since it will wait for any active
connections to close and would possible block the exit.
2023-12-18 10:53:34 +02:00
Tomi Turtiainen
6ae2f5efea fix(core): Fix shutdown if terminating before hooks are initialized (#8047)
If the app receives termination signal before hooks have been
initialised, the would be objet is undefined error. This PR fixes that.
2023-12-18 09:23:10 +02:00
Csaba Tuncsik
152883eed1 build: Upgrade lint related packages (#7790) 2023-11-27 09:11:52 +01:00
कारतोफ्फेलस्क्रिप्ट™
05586a900d refactor(core): Make Logger a service (no-changelog) (#7494) 2023-10-25 16:35:22 +02:00
Michael Auerswald
afa683a06f refactor(core): Have one orchestration service per instance type (#7303)
webhook instances will not listen to either worker or event log messages
on the Redis pub/sub channel
2023-10-06 13:58:11 +02:00
Omar Ajoue
5cfd87692f refactor: Add log messages to debug worker init issues (#7238)
Github issue / Community forum post (link here to close automatically):
2023-10-05 13:37:25 +02:00
Michael Auerswald
4b014286cf fix(core): Make senderId required for all command messages (#7252)
all commands sent between main instance and workers need to contain a
server id to prevent senders from reacting to their own messages,
causing loops

this PR makes sure all sent messages contain a sender id by default as
part of constructing a sending redis client.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-26 13:58:06 +02:00
Iván Ovejero
6d6e2488c6 refactor(core): Generalize binary data manager interface (no-changelog) (#7164)
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)

This PR: 
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.

Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.

Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Michael Auerswald
9f797b96d8 feat(core): Add command to trigger license refresh on workers (#7184)
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.

Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance

This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
Alex Grozav
ed927d34b2 feat: External Secrets storage for credentials (#6477)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-08-25 10:33:46 +02:00
Michael Auerswald
6499f42481 feat(core): Add unique id to instances (#6863)
* add serverId and workerId

* adjust id format

* fix logger init / test

* update implementation
2023-08-07 17:03:21 +02:00
Iván Ovejero
72523462ea refactor: Clear unused ESLint directives from BE packages (no-changelog) (#6798) 2023-07-31 11:00:48 +02:00
Omar Ajoue
42c79cd6f1 fix: Initialize license in queue mode correctly (#6301) 2023-05-23 13:01:50 +02:00
कारतोफ्फेलस्क्रिप्ट™
9e7b9fb443 feat(core): Improve health check (#6205)
* remove unnecesary Db re-initialization

this is from before we added `Db.init` in `WorkflowRunnerProcess`

* feat(core): Improved health check

* make health check not care about DB connections

* close DB connections, and shutdown the timer
2023-05-10 10:27:04 +02:00
कारतोफ्फेलस्क्रिप्ट™
e67ad29625 fix(core): Initialize queue in the webhook server as well (#5766)
fix(core): Initialize queue in the webhook server as well (no-changelog)
2023-03-24 10:47:21 +01:00
कारतोफ्फेलस्क्रिप्ट™
52f740b9e8 refactor(core): Use an IoC container to manage singleton classes [Part-1] (no-changelog) (#5509)
* add typedi

* convert ActiveWorkflowRunner into an injectable service

* convert ExternalHooks into an injectable service

* convert InternalHooks into an injectable service

* convert LoadNodesAndCredentials into an injectable service

* convert NodeTypes and CredentialTypes into an injectable service

* convert ActiveExecutions into an injectable service

* convert WaitTracker into an injectable service

* convert Push into an injectable service

* convert ActiveWebhooks and  TestWebhooks into an injectable services

* handle circular references, and log errors when a circular dependency is found
2023-02-21 19:21:56 +01:00
Jan Oberhauser
339655611f fix(core): Fix issue that worker and webhook service close directly (#5461) 2023-02-11 09:46:10 -06:00
कारतोफ्फेलस्क्रिप्ट™
9a331ec7b6 fix(core): Stop using unbound calls to stopProcess (no-changelog) (#5456) 2023-02-10 16:34:39 +01:00
कारतोफ्फेलस्क्रिप्ट™
5194513850 refactor(core): Delete duplicate code across all commands (#5452) 2023-02-10 14:59:20 +01:00
कारतोफ्फेलस्क्रिप्ट™
72249e0de8 refactor(core): Load and validate all config at startup (no-changelog) (#5283) 2023-01-30 14:42:30 +01:00
कारतोफ्फेलस्क्रिप्ट™
a573db2ef7 fix: Report app startup and DB migration errors to Sentry (#5127) 2023-01-11 18:28:35 +01:00
कारतोफ्फेलस्क्रिप्ट™
8b19fdd5f0 refactor: Delete a lot of unused and duplicate code in Server and WebhookServer (#5080)
* store n8n version string in a const and use that everywhere

* reduce code duplication between Server and WebhookServer

* unify redis checks

* fix linting
2023-01-04 11:38:48 +01:00
Thomas S
0ec66bfb42 feat(core): Add compatibility to redis > 6 ACLs system using username in queue-mode (#5048)
feat(redis): add compatibility to redis > 6 ACLs system using username

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-03 11:45:03 +01:00
कारतोफ्फेलस्क्रिप्ट™
b828cb31d6 perf: Lazy-load queue-mode and analytics dependencies (#5061)
* refactor: lazy load ioredis and bull

* upgrade bull and hiredis

* refactor: lazy load posthog, rudderstack, and sentry

* upgrade Sentry sdk
2023-01-02 12:14:39 +01:00
Iván Ovejero
d9b98fc8be refactor: Lint for no unneeded backticks (#5057) (no-changelog)
*  Create rule `no-unneeded-backticks`

* 👕 Enable rule

*  Run rule on `cli`

*  Run rule on `core`

*  Run rule on `workflow`

*  Rule rule on `design-system`

*  Run rule on `node-dev`

*  Run rule on `editor-ui`

*  Run rule on `nodes-base`
2022-12-29 12:20:43 +01:00