Skip to content

buffer: Invalid bytes length calculation #11165

@avchugaev

Description

@avchugaev

Node Version: v7.5.0
Platform: Windows 10 x64

It seems that Buffer.byteLength() method and buffer.byteLength returns different values for base64 encoding. Maybe for some others too.

Here is TypeScript class that works with encodings:

export type EncodingName = 'utf8' | 'utf16le' | 'ascii' | 'latin1' | 'ucs2' | 'hex' | 'base64' | 'binary';


export default class Encoding {
    public static UTF8: Encoding = new Encoding('utf8');
    public static UTF16LE: Encoding = new Encoding('utf16le');
    public static ASCII: Encoding = new Encoding('ascii');
    public static LATIN1: Encoding = new Encoding('latin1');
    public static UCS2: Encoding = new Encoding('ucs2');
    public static HEX: Encoding = new Encoding('hex');
    public static BASE64: Encoding = new Encoding('base64');
    public static BINARY: Encoding = new Encoding('binary');


    public static convert(originalString: string, originalEncoding: Encoding, targetEncoding: Encoding): string {
        return originalEncoding.getBytes(originalString).toString(targetEncoding.encodingName);
    }


    private _encodingName: EncodingName;


    get encodingName(): EncodingName {
        return this._encodingName;
    }


    constructor(encodingName: EncodingName = 'utf8') {
        this._encodingName = encodingName;
    }


    public getBytesCount(text: string): number {
        return Buffer.byteLength(text, this.encodingName);
    }


    public getBytes(text: string): Buffer {
        return Buffer.from(text, this.encodingName);
    }
}

let source: string = 'abc123 \u03C0';               // abc123 π
let base64String: string = Encoding.convert(source, Encoding.UTF8, Encoding.BASE64);
let originalString: string = Encoding.convert(base64String, Encoding.BASE64, Encoding.UTF8);

console.log(
    Encoding.UTF8.getBytes(source).byteLength,      // 9
    Encoding.UTF8.getBytesCount(source)             // 9
);

console.log(
    Encoding.BASE64.getBytes(source).byteLength,    // 5
    Encoding.BASE64.getBytesCount(source)           // 6  - Why different length?
);

console.log(originalString);                        // abc123 π
console.log(base64String);                          // YWJjMTIzIM+A

Metadata

Metadata

Assignees

No one assigned

    Labels

    bufferIssues and PRs related to the buffer subsystem.docIssues and PRs related to the documentations.good first issueIssues that are suitable for first-time contributors.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions