Warning This package has been deprecated. We have found a faster, simpler and more reliable way. See issue #12.

GenRawID

genrawid is a niche command line tool and/or Go package to generate a unique consistent number from the input.

$ genrawid --string "abcdefgh"
-2474118025671277174
$ # 'abcdefgh'           --> The data to store in SQLite3
$ # -2474118025671277174 --> The primary key/rawid of stored data

$ echo -n 'abcdefgh' > ./sample.txt
$ genrawid ./sample.txt
-2474118025671277174
$ # 'abcdefgh'           --> The data to store in SQLite3
$ # -2474118025671277174 --> The primary key/rawid of stored data

genrawid is similar to a hash function, but its value is a combination of a hash and a checksum.

The hash value is the digest of the input and the checksum is the CRC of the complete hash value.

By default, BLAKE3-512 is used for the hash algorithm and CRC-32C (CRC-32 with Castagnoli polynomial) for the checksum.

8 Bytes = The first 4 Bytes of the hash + 4 Bytes of the checksum of the hash

• See benchmark of BLAKE3 and CRC-32 comparing to other hash algorithms:
  • https://github.com/KEINOS/go-blake3-example/blob/main/bench_results/bench_results_stats.txt

Why?

The main objective is to use SQLite3 as a fast KVS (Key-Value-Store) for CAS (Content-Addressable-Storage) usage.

genrawid generates a 64-bit/8-byte signed decimal number that can be used as rawid in SQLite3.

Searching for a record with a specific rowid, or for all records with rowids within a specified range is around twice as fast as a similar search made by specifying any other PRIMARY KEY or indexed value.

_{^{(From "ROWIDs and the INTEGER PRIMARY KEY" @ sqlite.org)}}

So, in theory, if you know the rawid of the content, you can find it twice as fast.

On the other hand, this command/package is of little use when using SQLite3 as RDB (Relational Database) or when dealing with mutable content.

Install

Download the binary for your operating system and architecture and place it as an executable in your PATH.

• Latest Releases (Windows, macOS, Linux, RaspberryPi)

For Homebrew/Linuxbrew users:

brew install KEINOS/apps/genrawid

Statuses

• Unit Tests/Code Coverage

• Secrurity/Vulnerability Check

CONTRIBUTING

• GolangCI Lint rules: .golangci-lint.yml
• To run tests in a container:
  • docker-compose --file ./.github/docker-compose.yml run v1_17
  • This will run:
    • go test -cover -race ./...
    • golangci-lint run
    • golint ./...
• Branch to PR: main
  • It is recommended that DraftPR be done first to avoid duplication of work.

License

• MIT, Copyright (c) KEINOS and the GenRawID contributors.