Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
0bdc0da
Changed winston-loggly to winston-loggly-bulk
Jun 14, 2016
9fa1309
Merge pull request #1 from psquickitjayant/master
Jun 14, 2016
d28c040
Added node analytics
varshneyjayant Jun 16, 2016
93b1b91
Merge pull request #2 from psquickitjayant/master
Jun 16, 2016
9e0d107
dep(winston-loggly.js) update dep
Shwetajain148 Oct 4, 2016
2d93629
Merge pull request #3 from Shwetajain148/update-dep
Oct 4, 2016
a680b38
pass loggly http client errors to winston continuation so they get pr…
zachgrayio Oct 7, 2016
eeaea5a
update package.json
Shwetajain148 Oct 7, 2016
359d5d3
Rename and update package references
Shwetajain148 Oct 7, 2016
6c91cb5
Merge pull request #5 from Shwetajain148/create-node-package
Oct 19, 2016
5c9e80f
update readme and package.json file
Shwetajain148 Nov 10, 2016
c7c127b
Merge pull request #6 from Shwetajain148/npm-publish-1.3.4
Nov 10, 2016
df209c1
update the email address for loggly
Shwetajain148 Nov 11, 2016
91f32dd
Merge pull request #8 from Shwetajain148/update_email
mostlyjason Nov 11, 2016
f207d67
queue support in library "winston-loggly-bulk" and make buffer option…
Shwetajain148 Jan 9, 2017
e8c4ed2
Merge pull request #9 from Shwetajain148/buffer-log-support
Jan 17, 2017
77b183f
update readme.md file
Shwetajain148 Feb 6, 2017
2908e2b
Merge pull request #11 from Shwetajain148/update-readme
Feb 7, 2017
0a7af4f
incude timestamp in nodejs events
Mar 28, 2017
e8f39b8
set timestamp field true by default and also make it configurable
Mar 29, 2017
72406c7
Merge pull request #14 from Shwetajain148/include-timestamp-in-events
Mar 29, 2017
3a23011
Update reademe file
Apr 12, 2017
53a48f4
update package version as per npm package
Apr 12, 2017
437f0d7
Merge pull request #16 from Shwetajain148/update-readme-file
mostlyjason Apr 25, 2017
8de2e3d
update version of node-loggly-bulk
May 22, 2017
f4f3ec5
include winston into dependencies and version update
May 24, 2017
46ffdbc
reverted winston version
May 25, 2017
32948de
Merge pull request #18 from Shwetajain148/update-node-version
mostlyjason Jun 6, 2017
5603d61
upgrade to version 2
Jun 7, 2017
2052c38
Merge pull request #20 from Shwetajain148/upgrade-to-version2
Jun 7, 2017
a470d07
handle undefined error
Jul 11, 2017
9f210fe
Merge pull request #21 from Shwetajain148/resolve-tags-of-undefined
Jul 13, 2017
c1ad2ee
Merge pull request #4 from scalio/master
mostlyjason Jul 25, 2017
4a2770c
helper method to call process exit
Aug 8, 2017
b08406d
update README.md file for flushLogsAndExit function
Aug 9, 2017
4caebd2
Update README.md
mostlyjason Aug 9, 2017
4d24722
Update README.md
mostlyjason Aug 9, 2017
e35aaae
Merge pull request #23 from Shwetajain148/expose-helper-method
Aug 16, 2017
134348d
Merge pull request #24 from Shwetajain148/update-readme-for-process-exit
Aug 16, 2017
1856f6d
update package.json file for version upgrade
Sep 22, 2017
02c7374
Merge pull request #25 from Shwetajain148/winston-loggly-bulk-version…
mostlyjason Sep 22, 2017
9434da4
Update instructions for npm version 2 and lower
Nov 29, 2017
79ad087
Merge pull request #28 from Shwetajain148/update-readme-for-npm-version2
mostlyjason Nov 29, 2017
58461d1
Error message support in winston-loggly-bulk
Jan 2, 2018
5e9dbcf
Merge pull request #30 from Shwetajain148/handling-error-object
Jan 3, 2018
dda95d0
Update package version
Jan 3, 2018
940a046
Merge pull request #31 from Shwetajain148/update-readme
mostlyjason Jan 3, 2018
5ab51dd
remove winston.clone
Jun 15, 2018
be3a4f9
Merge pull request #35 from Shwetajain148/3.x
Jun 19, 2018
7894a81
Update package.json dependencies
Jun 19, 2018
c4c541a
Merge pull request #37 from Shwetajain148/3.x
Jun 19, 2018
f4eaf39
Update package.json dependencies
Jun 19, 2018
a4847ef
Merge branch '3.x' into 3.x
Jun 19, 2018
1a060bc
Merge pull request #38 from Shwetajain148/3.x
Jun 19, 2018
077c48b
Fixed #49
grantcarthew May 25, 2017
7cd179b
Added package-lock
OtterCode Jun 20, 2018
ad9ed3a
Updated transport for winston 3.0
OtterCode Jun 20, 2018
062d97b
Added clone to replace winston.clone
OtterCode Jun 21, 2018
15b63b1
Modified test suite and updated deps.
OtterCode Jun 21, 2018
180e401
Small fixes.
OtterCode Jun 25, 2018
0deb9cf
Package versions
OtterCode Jun 28, 2018
a798f37
Merge branch 'master' into master
DABH Sep 26, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 77 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,25 @@
# winston-loggly
# winston-loggly-bulk

A [Loggly][0] transport for [winston][1].

[![Version npm](https://img.shields.io/npm/v/winston-loggly-bulk.svg?style=flat-square)](https://www.npmjs.com/package/winston-loggly-bulk)[![npm Downloads](https://img.shields.io/npm/dm/winston-loggly-bulk.svg?style=flat-square)](https://www.npmjs.com/package/winston-loggly-bulk)

[![NPM](https://nodei.co/npm/winston-loggly-bulk.png?downloads=true&downloadRank=true)](https://nodei.co/npm/winston-loggly-bulk/)

A client implementation for Loggly in node.js. Check out Loggly's [Node logging documentation](https://www.loggly.com/docs/nodejs-logs/) for more.


## Usage
``` js
var winston = require('winston');

//
// Requiring `winston-loggly` will expose
// Requiring `winston-loggly-bulk` will expose
// `winston.transports.Loggly`
//
require('winston-loggly');
var {Loggly} = require('winston-loggly-bulk');

winston.add(winston.transports.Loggly, options);
winston.add(new Loggly({options}));
```

The Loggly transport is based on [Nodejitsu's][2] [node-loggly][3] implementation of the [Loggly][0] API. If you haven't heard of Loggly before, you should probably read their [value proposition][4]. The Loggly transport takes the following options. Either 'inputToken' or 'inputName' is required:
Expand All @@ -26,10 +33,58 @@ The Loggly transport is based on [Nodejitsu's][2] [node-loggly][3] implementatio
* __tags:__ An array of tags to send to loggly.
* __isBulk:__ If true, sends messages using bulk url
* __stripColors:__ Strip color codes from the logs before sending

* __bufferOptions:__ Buffer options has two configurations.
- __size:__ Number of logs to be buffered.
- __retriesInMilliSeconds:__ Time in milliseconds to retry sending buffered logs.
* __timestamp:__ If false, library will not include timestamp in log events.
- __Note:__ Library includes timestamp by default when we do not set timestamp option.

*Metadata:* Logged in suggested [Loggly format][5]

## Sample Working Code Snippet

``` js
var winston = require('winston');
var {Loggly} = require('winston-loggly-bulk');

winston.add(new Loggly({
token: "TOKEN",
subdomain: "SUBDOMAIN",
tags: ["Winston-NodeJS"],
json: true
}));

winston.log('info', "Hello World from Node.js!");
```

## Buffer Support

This library has buffer support during temporary network outage. User can configure size of buffer (no. of logs to be stored during network outage).

Add these below configuration in code snippet to override the default values of buffer option __size__ and __retriesInMilliSeconds__.
``` js
bufferOptions: {
size: 1000,
retriesInMilliSeconds: 60 * 1000
}
```
* __Note:__ The default value of buffer size and retries in milliseconds are 500 and 30000 (30 seconds) respectively.

## Flush logs and exit

Our library uses ajax requests to send logs to Loggly, and as ajax requests take time to complete, logs can be lost when process.exit() is called because it forces an immediate exit. To exit gracefully and ensure that the last logs get to Loggly, we created a function called flushLogsAndExit(). It waits for 10 seconds and then calls process.exit() itself. This allows enough time for the logs to be sent to Loggly.

Here is an example of how to use the method:

``` js
var winston = require('winston');
var {flushLogsAndExit} = require('winston-loggly-bulk');

winston.log("info", "Hello World from Node.js!");
flushLogsAndExit();

```

## Motivation
`tldr;?`: To break the [winston][1] codebase into small modules that work together.

Expand All @@ -43,11 +98,23 @@ The [winston][1] codebase has been growing significantly with contributions and
$ curl http://npmjs.org/install.sh | sh
```

### Installing winston-loggly
### Installing winston-loggly-bulk

If you are running npm version 3 or higher then run the below command to setup the logging-

``` bash
$ npm install winston-loggly-bulk
```

If you are running npm version 2 or lower then run the below command to setup the logging-

``` bash
$ npm install winston-loggly-bulk winston
```
* __Note:__ To check the currrent npm version run the below command-

``` bash
$ npm install winston
$ npm install winston-loggly
$ npm -v
```

## Run Tests
Expand All @@ -74,8 +141,8 @@ Once you have valid configuration and credentials you can run tests with [npm][7
npm test
```

#### Author: [Charlie Robbins](http://blog.nodejitsu.com)
#### License: MIT
#### Author: [Charlie Robbins](http://www.github.com/indexzero)
#### Contributors: [Loggly](http://github.com/loggly), [Shweta Jain](http://github.com/shwetajain148),

[0]: http://loggly.com
[1]: https://github.com/flatiron/winston
Expand Down
72 changes: 50 additions & 22 deletions lib/winston-loggly.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,20 @@
*
*/

var events = require('events'),
loggly = require('loggly'),
var clone = require('clone'),
loggly = require('node-loggly-bulk'),
util = require('util'),
winston = require('winston'),
Transport = require('winston-transport'),
Stream = require('stream').Stream;

//
// Remark: This should be at a higher level.
//
var code = /\u001b\[(\d+(;\d+)*)?m/g;

var timerFunctionForProcessExit = null;

//
// ### function Loggly (options)
// #### @options {Object} Options for this instance.
Expand All @@ -33,7 +36,7 @@ var Loggly = exports.Loggly = function (options) {
options.token = options.inputToken;
}

winston.Transport.call(this, options);
Transport.call(this, options);
if (!options.subdomain) {
throw new Error('Loggly Subdomain is required');
}
Expand All @@ -50,33 +53,57 @@ var Loggly = exports.Loggly = function (options) {
this.client = loggly.createClient({
subdomain: options.subdomain,
auth: options.auth || null,
json: options.json || false,
json: options.json || false, //TODO: should be false
proxy: options.proxy || null,
token: options.token,
tags: tags,
isBulk: options.isBulk || false
isBulk: options.isBulk || false,
bufferOptions: options.bufferOptions || {size: 500, retriesInMilliSeconds: 30 * 1000}
});

this.timestamp = options.timestamp || false;
this.timestamp = options.timestamp === false ? false : true;
this.stripColors = options.stripColors || false;
};

//
// Helper function to call process.exit() after waiting
// 10 seconds.
//
var flushLogsAndExit = exports.flushLogsAndExit = function () {
if (timerFunctionForProcessExit === null) {
timerFunctionForProcessExit = setInterval(function () {
process.exit();
},10000);
}
}

//
// Inherit from `winston.Transport`.
//
util.inherits(Loggly, winston.Transport);
util.inherits(Loggly, Transport);

//
// Define a getter so that `winston.transports.Loggly`
// is available and thus backwards compatible.
//
winston.transports.Loggly = Loggly;
winston.transports.flushLogsAndExit = flushLogsAndExit;

//
// Expose the name of this Transport on the prototype
//
Loggly.prototype.name = 'loggly';

const validateMetadata = (meta) => {
if (meta == null) {
return {};
} else if (typeof meta !== 'object') {
return { metadata: meta };
} else {
return clone(meta);
}
}

//
// ### function log (level, msg, [meta], callback)
// #### @level {string} Level at which to log the message.
Expand All @@ -86,34 +113,35 @@ Loggly.prototype.name = 'loggly';
// Core logging method exposed to Winston. Metadata is optional.
//
Loggly.prototype.log = function (level, msg, meta, callback) {

const message = validateMetadata(meta);

if (this.silent) {
return callback(null, true);
}

if (this.timestamp && (!meta || !meta.timestamp)) {
meta = meta || {};
meta.timestamp = (new Date()).toISOString();
if (this.timestamp && !message.timestamp) {
message.timestamp = (new Date()).toISOString();
}

if (this.stripColors) {
msg = ('' + msg).replace(code, '');
}

var message = winston.clone(meta || {}),
self = this;
const self = this;

message.level = level;
message.message = msg;
message.message = msg || message.message;

//
// Helper function for responded to logging.
//
function logged() {
function logged(err) {
self.emit('logged');
callback(null, true);
callback(err, true);
}

return meta && meta.tags
return (meta && meta.tags)
? this.client.log(message, meta.tags, logged)
: this.client.log(message, logged);
};
Expand All @@ -123,9 +151,9 @@ Loggly.prototype.log = function (level, msg, meta, callback) {
// #### @options {Object} Set stream options
// Returns a log stream.
//
Loggly.prototype.stream = function(options) {
Loggly.prototype.stream = function(maybeOptions) {
var self = this,
options = options || {},
options = maybeOptions || {},
stream = new Stream,
last,
start = options.start,
Expand Down Expand Up @@ -157,15 +185,15 @@ Loggly.prototype.stream = function(options) {
return setTimeout(check, 2000);
}

var result = res[res.length-1];
var result = results[results.length-1];
if (result && result.timestamp) {
if (last == null) {
last = result.timestamp;
return;
}
last = result.timestamp;
} else {
return func();
return;
}

results.forEach(function(log) {
Expand Down Expand Up @@ -218,10 +246,10 @@ Loggly.prototype.formatQuery = function (query) {
// ### function formatResults (results, options)
// #### @results {Object|Array} Results returned from `.query`.
// #### @options {Object} **Optional** Formatting options
// Formats the specified `results` with the given `options` accordinging
// Formats the specified `results` with the given `options` according
// to the implementation of this transport.
//
Loggly.prototype.formatResults = function (results, options) {
Loggly.prototype.formatResults = function (results, _options) {
return results;
};

Expand Down
Loading