Commit Graph

52 Commits

Author SHA1 Message Date
Omar Ajoue
cdbb3acd2d 🐛 Remove webhooks deregistration from webhook process (#1753) 2021-05-07 19:37:45 -05:00
Omar Ajoue
c972f3dd50 Added logging to n8n (#1381)
* Added logging to n8n

This commit adds logging to n8n using the Winston library.

For now, this commit only allows logging to console (default behavior)
or file (need to pass in config via environment variables).

Other logging methods can be further implemented using hooks. These were
skipped for now as it would require adding more dependencies.

Logging level is notice by default, meaning no additional messages would
be displayed at the moment. Logging level can be set to info or debug as
well to enrich the generated logs.

The ILogger interface was added to the workflow project as it would make
it available for all other projects but the implementation was done on
the cli project.

* Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously

* Improvements to logging framework

Using appropriate single quotes
Improving the way the logger is declared

* Improved naming for Log Types

* Removed logger global variable, replacing it by a proxy

* Add logging to CLI commands

* Remove unused GenericHelpers

* Changed back some messages to console instead of logger and added npm
shortcuts for worker and webhook

* Fix typos

* Adding basic file rotation to logs as suggested by @mutdmour

* Fixed linting issues

* Correcting comment to correctly reflect space usage

* Added settings for log files rotation

* Correcting config type from String to Number

* Changed default file settings to number

To reflect previous changes to the type

* Changed the way log messages are added to be called statically. Also minor naming improvements

* Applying latest corrections sent by @ivov

*  Some logging improvements

* Saving logs to a folder inside n8n home instead of root

* Fixed broken tests and linting

* Changed some log messages to improve formatting

* Adding quotes to names  on log messages

* Added execution and session IDs to logs. Also removed unnecessary line breaks

*  Added file caller to log messages (#1657)

This is done using callsites library which already existed
in the project as another library's dependency. So in fact
it does not add any new dependency.

* Adding logs to help debug Salesforce node

*  Add function name to logs and add more logs

*  Improve some error messages

*  Improve some more log messages

*  Rename logging env variables to match others

Co-authored-by: dali <servfrdali@yahoo.fr>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 22:43:01 -05:00
Omar Ajoue
0c779de704 Implement timeout for workers and corrected timeout for subworkflows (#1634)
* Implemented timeout for workers and corrected timeout for subworkflows

* Fixed issue with timeouts when running on separate processes

* Standardized timeouts across all n8n

Now the maxTimeout setting takes effect whenever a timeout is set
to any workflows.

This causes local timeouts (either set on a per-workflow basis or
via global settings) to be capped by the maximum timeout. This
behavior already existed but was not applied to all places.

Also changed the way n8n enforces timeouts for subworkflows, making
it work always.

In effect, with this change, if you have one workflow that calls others
only the main workflow's timeout is taken into consideration, limiting
the maximum time that other workflows combined can run.

*  Fix timeout problem in "own" mode

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-04-17 16:44:07 +02:00
Iván Ovejero
1d27a9e87e Improve node error handling (#1309)
* Add path mapping and response error interfaces

* Add error handling and throwing functionality

* Refactor error handling into a single function

* Re-implement error handling in Hacker News node

* Fix linting details

* Re-implement error handling in Spotify node

* Re-implement error handling in G Suite Admin node

* 🚧 create basic setup NodeError

* 🚧 add httpCodes

* 🚧 add path priolist

* 🚧 handle statusCode in error, adjust interfaces

* 🚧 fixing type issues w/Ivan

* 🚧 add error exploration

* 👔 fix linter issues

* 🔧 improve object check

* 🚧 remove path passing from NodeApiError

* 🚧 add multi error + refactor findProperty method

* 👔 allow any

* 🔧 handle multi error message callback

*  change return type of callback

*  add customCallback to MultiError

* 🚧 refactor to use INode

* 🔨 handle arrays, continue search after first null property found

* 🚫 refactor method access

* 🚧 setup NodeErrorView

*  change timestamp to Date.now

* 📚 Add documentation for methods and constants

* 🚧 change message setting

* 🚚 move NodeErrors to workflow

*  add new ErrorView for Nodes

* 🎨 improve error notification

* 🎨 refactor interfaces

*  add WorkflowOperationError, refactor error throwing

* 👕 fix linter issues

* 🎨 rename param

* 🐛 fix handling normal errors

*  add usage of NodeApiError

* 🎨 fix throw new error instead of constructor

* 🎨 remove unnecessary code/comments

* 🎨 adjusted spacing + updated status messages

* 🎨 fix tab indentation

*  Replace current errors with custom errors (#1576)

*  Introduce NodeApiError in catch blocks

*  Introduce NodeOperationError in nodes

*  Add missing errors and remove incompatible

*  Fix NodeOperationError in incompatible nodes

* 🔧 Adjust error handling in missed nodes

PayPal, FileMaker, Reddit, Taiga and Facebook Graph API nodes

* 🔨 Adjust Strava Trigger node error handling

* 🔨 Adjust AWS nodes error handling

* 🔨 Remove duplicate instantiation of NodeApiError

* 🐛 fix strava trigger node error handling

* Add XML parsing to NodeApiError constructor (#1633)

* 🐛 Remove type annotation from catch variable

*  Add XML parsing to NodeApiError

*  Simplify error handling in Rekognition node

*  Pass in XML flag in generic functions

* 🔥 Remove try/catch wrappers at call sites

* 🔨 Refactor setting description from XML

* 🔨 Refactor let to const in resource loaders

*  Find property in parsed XML

*  Change let to const

* 🔥 Remove unneeded try/catch block

* 👕 Fix linting issues

* 🐛 Fix errors from merge conflict resolution

*  Add custom errors to latest contributions

* 👕 Fix linting issues

*  Refactor MongoDB helpers for custom errors

* 🐛 Correct custom error type

*  Apply feedback to A nodes

*  Apply feedback to missed A node

*  Apply feedback to B-D nodes

*  Apply feedback to E-F nodes

*  Apply feedback to G nodes

*  Apply feedback to H-L nodes

*  Apply feedback to M nodes

*  Apply feedback to P nodes

*  Apply feedback to R nodes

*  Apply feedback to S nodes

*  Apply feedback to T nodes

*  Apply feedback to V-Z nodes

*  Add HTTP code to iterable node error

* 🔨 Standardize e as error

* 🔨 Standardize err as error

*  Fix error handling for non-standard nodes

Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>

Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
2021-04-16 18:33:36 +02:00
Jan Oberhauser
0c0c0bda67 Some cleanup 2021-03-25 11:23:54 +01:00
Jan Oberhauser
09d9e12db9 🐛 Fix issue with importing decrypted separate credentials #1546 2021-03-17 21:28:22 +01:00
Omar Ajoue
8095c4a264 Add debug information for pending workflows on exit (#1536) 2021-03-12 11:35:23 +01:00
Omar Ajoue
d6d57c2df6 🐛 Fixed an issue with queue mode for executions that should not be saved (#1509)
* Fixed an issue with queue mode for executions that should not be saved

*  Minimal change

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-03-10 15:51:18 +01:00
Jan Oberhauser
802808a9ad Minor improvement 2021-02-21 22:37:39 +01:00
Allan Daemon
7eb849b9f7 CLI import credentials supports decrypted data 2021-02-18 07:52:05 -03:00
Allan Daemon
873e566489 CLI command export credentials: export decrypted option 2021-02-18 07:42:21 -03:00
Omar Ajoue
e53efdd337 Separate webhooks from core (#1408)
* Unify execution ID across executions

* Fix indentation and improved comments

* WIP: saving data after each node execution

* Added on/off to save data after each step, saving initial data and retries working

* Fixing lint issues

* Fixing more lint issues

*  Add bull to execute workflows

* 👕 Fix lint issue

*  Add graceful shutdown to worker

*  Add loading staticData to worker

* 👕 Fix lint issue

*  Fix import

* Changed tables metadata to add nullable to stoppedAt

* Reload database on migration run

* Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration

* Added checks to Redis and exiting process if connection is unavailable

* Fixing error with new installations

* Fix issue with data not being sent back to browser on manual executions with defined destination

* Merging bull and unify execution id branch fixes

* Main process will now get execution success from database instead of redis

* Omit execution duration if execution did not stop

* Fix issue with execution list displaying inconsistant information information while a workflow is running

* Remove unused hooks to clarify for developers that these wont run in queue mode

* Added active pooling to help recover from Redis crashes

* Lint issues

* Changing default polling interval to 60 seconds

* Removed unnecessary attributes from bull job

* Added webhooks service and setting to disable webhooks from main process

* Fixed executions list when running with queues. Now we get the list of actively running workflows from bull.

* Add option to disable deregistration of webhooks on shutdown

* Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat.

* Added auto refresh to executions list

* Improvements to workflow stop process when running with queues

* Refactor queue system to use a singleton and avoid code duplication

* Improve comments and remove unnecessary commits

* Remove console.log from vue file

* Blocking webhook process to run without queues

* Handling execution stop graciously when possible

* Removing initialization of all workflows from webhook process

* Refactoring code to remove code duplication for job stop

* Improved execution list to be more fluid and less intrusive

* Fixing workflow name for current executions when auto updating

*  Right align autorefresh checkbox

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 23:32:40 +01:00
Omar Ajoue
7a3aaf8a24 Unify execution id + Queue system (#1340)
* Unify execution ID across executions

* Fix indentation and improved comments

* WIP: saving data after each node execution

* Added on/off to save data after each step, saving initial data and retries working

* Fixing lint issues

* Fixing more lint issues

*  Add bull to execute workflows

* 👕 Fix lint issue

*  Add graceful shutdown to worker

*  Add loading staticData to worker

* 👕 Fix lint issue

*  Fix import

* Changed tables metadata to add nullable to stoppedAt

* Reload database on migration run

* Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration

* Added checks to Redis and exiting process if connection is unavailable

* Fixing error with new installations

* Fix issue with data not being sent back to browser on manual executions with defined destination

* Merging bull and unify execution id branch fixes

* Main process will now get execution success from database instead of redis

* Omit execution duration if execution did not stop

* Fix issue with execution list displaying inconsistant information information while a workflow is running

* Remove unused hooks to clarify for developers that these wont run in queue mode

* Added active pooling to help recover from Redis crashes

* Lint issues

* Changing default polling interval to 60 seconds

* Removed unnecessary attributes from bull job

*  Improved output on worker job start

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 08:59:32 +01:00
Jan Oberhauser
910c0eaa7a Improve CLI export/import and add missing files 2021-01-21 10:52:33 +01:00
Jan Oberhauser
144de3f24b 👕 Fix lint issue 2021-01-21 10:47:45 +01:00
Omar Ajoue
cd7ed7db47 Add CLI export/import for credentials/workflows (#1363)
* Added options to import and export workflows and credentials

* Fixed linting issues and descriptions

* Linting issues and allowing import/export operations to output multiple files

*  Minor changes to CLI export/import

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-01-20 23:51:49 +01:00
Omar Ajoue
ba8ea6c0f5 Removing console.log message 2021-01-06 13:01:56 +01:00
Omar Ajoue
04a8912a94 Changed vacuum operation to run on startup only. Also it is now off by default. 2021-01-04 09:30:02 +01:00
Omar Ajoue
d528e442ba Run vacuum on sqlite periodically 2020-12-31 10:42:16 +01:00
Jan Oberhauser
ecdcc0b522 🐛 Fix issue that DB init errors did not get caught #1269 2020-12-24 09:06:43 +01:00
Ben Hesseldieck
f2666e92ff Add preExecuteHooks (#1151)
*  Save initital data on hook error

* 🚧 update function interface

* 🚧 response webhook with error, 🐛 fix adding preExecutionHooks to hooks

* 🔥 remove execute hook

*  execute preExecute hooks on integrated workflows

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2020-11-13 23:31:27 +01:00
Jan Oberhauser
ac346b125b Change CLI command config -> update 2020-10-27 11:14:51 +01:00
Jan Oberhauser
205a033717 Add CLI command to deactivate workflows 2020-10-24 14:37:45 +02:00
Jan Oberhauser
40c2acd77b 👕 Fix lint issue 2020-10-22 15:46:03 +02:00
Jan Oberhauser
df020136d3 Make sure that n8n stops even if there are unforeseen errors 2020-10-14 16:41:42 +02:00
Ben Hesseldieck
28074d94a4 Add hooks (#1053)
*  add 2 hooks

*  Fix issue that if hook errors n8n would not shut down

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2020-10-14 16:38:48 +02:00
Jan Oberhauser
8366632268 🐛 Fix bug with missing credentials with CLI execute 2020-10-10 18:23:02 +02:00
Jan Oberhauser
397132688e Wait for active workflow to finish before shutting down 2020-09-14 12:30:58 +02:00
Rupenieks
a69098a096 Updated @types/node to v14, fixed TS breaking with changes 2020-08-12 12:19:41 +02:00
Jan Oberhauser
1451f94814 🔀 Merge branch 'external-hooks' 2020-06-10 13:02:45 +02:00
Jan Oberhauser
57b3a2e05e Cleanup of external hooks 2020-06-10 13:00:28 +02:00
Jan Oberhauser
a380a9a394 Add first basic code for external hooks 2020-05-04 08:56:01 +02:00
Jan Oberhauser
9dd9e0d8ba 🔀 Merge branch 'master' into oauth-support 2020-04-04 17:34:10 +02:00
Frane Bandov
b6b63ed48b Add full-icu to Dockerfiles 2020-03-26 18:05:23 +01:00
Jan Oberhauser
f145c499c0 If trigger nodes are in workflow use them as default start-node #379 2020-03-16 09:59:29 +01:00
Jan Oberhauser
eb285ef711 Make it possible to set credentials to fixed values 2020-01-25 23:48:38 -08:00
Jan Oberhauser
70e972ff7d Revert "🔀 Merge branch 'oauth-support' of https://github.com/tildabio/n8n"
This reverts commit 0c5972bb98, reversing
changes made to 3de03b5097.
2020-01-10 14:26:06 -06:00
Ram Yalamanchili
ed93611f43 Fix issue with tracking changes to rest api code in dev mode 2020-01-01 22:55:16 -08:00
Jan Oberhauser
da425ff857 ⬆️ Set localtunnel@2.0.0 on n8n 2019-12-29 12:22:37 -06:00
Jan Oberhauser
a088054602 🐛 Remove internationalization support because of docker build issues 2019-11-23 23:23:06 +01:00
Jan Oberhauser
60241d3163 Add support for internationalization 2019-11-23 11:43:00 +01:00
Jan Oberhauser
369e8084d3 Make it possible to set tunnel subdomain via environment variable 2019-10-15 07:21:15 +02:00
Jan Oberhauser
d53ab0bb6e Remove no longer needed restriction about process id 1 2019-09-03 13:42:43 +02:00
Jan Oberhauser
67afc883db Replace vorpal with oclif in cli 2019-08-28 15:28:47 +02:00
Jan Oberhauser
45721c20ca 📚 Make clear that tunnel is only for testing&development #13 2019-08-22 16:45:00 +02:00
Jan Oberhauser
d59a043e3f Run workflows in own independent subprocess 2019-08-08 20:47:08 +02:00
Jan Oberhauser
f3d84fc29e Make it possible to secure n8n via basic auth 2019-08-04 14:24:48 +02:00
Jan Oberhauser
d027545986 Use convict for configuration to make n8n easier to configure 2019-07-21 19:52:57 +02:00
Jan Oberhauser
034c70f6ab Make n8n work with older node versions (<10) 2019-06-24 12:47:44 +02:00
Jan Oberhauser
2b7382cd63 Rename cli command "run" to "execute" 2019-06-24 08:28:24 +02:00