Skip to content

Commit 66f82b1

Browse files
rusherrusher
committed
[misc] exact number implementation compatibility correction
1 parent f65e0ea commit 66f82b1

File tree

13 files changed

+378
-135
lines changed

13 files changed

+378
-135
lines changed

‎CHANGELOG.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,35 @@
11
# Change Log
22

3+
## [3.0.0-beta](https://github.com/mariadb-corporation/mariadb-connector-nodejs/tree/3.0.0-beta) (08 Jun 2021)
4+
[Full Changelog](https://github.com/mariadb-corporation/mariadb-connector-nodejs/compare/2.5.4...3.0.0-beta)
5+
6+
Migrating from 2.x or mysql/mysql2 driver have some breaking changes, see [dedicated part](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/maintenance/3.x/documentation/promise-api.md#migrating-from-2x-or-mysqlmysql2-to-3x) documentation.
7+
8+
* [CONJS-153] support Prepared statement with 10.6 new feature metadata skip
9+
* [CONJS-165] Adding initial message error value on Error object
10+
* [CONJS-166] Restrict authentication plugin list
11+
* [CONJS-167] Permit custom logger configuration
12+
13+
New Connection options
14+
15+
|option|description|type|default|
16+
|---:|---|:---:|:---:|
17+
| **insertIdAsNumber** | Whether the query should return last insert id from INSERT/UPDATE command as BigInt or Number. default return BigInt |*boolean* | false |
18+
| **decimalAsNumber** | Whether the query should return decimal as Number. If enable, this might return approximate values. |*boolean* | false |
19+
| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enable, this might return approximate values. |*boolean* | false |
20+
| **logger** | Permit custom logger configuration. For more information, see the [`logger` option](#logger) documentation. |*mixed*|
21+
| **prepareCacheLength** | Define prepare LRU cache length. 0 means no cache |*int*| 256 |
22+
23+
new Connection methods
24+
* [`connection.prepare(sql) → Promise`](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#connectionpreparesql---promise): Prepares a query.
25+
* [`connection.execute(sql[, values]) → Promise`](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#connectionexecutesql-values--promise): Prepare and Executes a query.
26+
27+
This methods are compatible with mysql2 with some differences:
28+
* permit streaming parameters
29+
* execute use by default a prepared cache that hasn't infinite length.
30+
* implement mariadb 10.6 skipping metadata when possible for better performance
31+
* Doesn't have a unprepare methods.
32+
333
## [2.5.4](https://github.com/mariadb-corporation/mariadb-connector-nodejs/tree/2.5.4) (08 Jun 2021)
434
[Full Changelog](https://github.com/mariadb-corporation/mariadb-connector-nodejs/compare/2.5.3...2.5.4)
535

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ See [promise documentation](https://github.com/mariadb-corporation/mariadb-conne
2525

2626
[Callback documentation](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/callback-api.md) describe the callback wrapper for compatibility with existing drivers.
2727

28-
See [dedicated part](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#migrating-from-2.x-or-mysql/mysql2-to-3.x) for migration from mysql/mysql2 or from 2.x version.
28+
See [dedicated part](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#migrating-from-2x-or-mysqlmysql2-to-3x) for migration from mysql/mysql2 or from 2.x version.
2929

3030

3131
## Why a New Client?

‎documentation/connection-options.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,8 @@
2828
| **insertIdAsNumber** | Whether the query should return last insert id from INSERT/UPDATE command as BigInt or Number. default return BigInt |*boolean* | false |
2929
| **decimalAsNumber** | Whether the query should return decimal as Number. If enable, this might return approximate values. |*boolean* | false |
3030
| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enable, this might return approximate values. |*boolean* | false |
31-
| **logger** | Configure logger. For more information, see the [`logger` option](#logger) documentation. |*mixed*|
31+
| **logger** | Permit custom logger configuration. For more information, see the [`logger` option](#logger) documentation. |*mixed*|
32+
| **prepareCacheLength** | Define prepare LRU cache length. 0 means no cache |*int*| 256 |
3233

3334
### JSON or String configuration
3435

@@ -389,6 +390,8 @@ mariadb.createConnection({
389390
| **cachingRsaPublicKey** | Indicate path/content to MySQL server caching RSA public key. use requires Node.js v11.6+ |*string* | |
390391
| **allowPublicKeyRetrieval** | Indicate that if `rsaPublicKey` or `cachingRsaPublicKey` public key are not provided, if client can ask server to send public key. |*boolean* | false |
391392
| **restrictedAuth** | if set, restrict authentication plugin to secure list. Default provided plugins are mysql_native_password, mysql_clear_password, client_ed25519, dialog, sha256_password and caching_sha2_password |*Array|String* | |
393+
| **supportBigNumbers** | (deprecated) DECIMAL/BIGINT data type will be returned as number if in safe integer range, as string if not.|*boolean* | false |
394+
| **bigNumberStrings** | (deprecated) if set with `supportBigNumbers` DECIMAL/BIGINT data type will be returned as string |*boolean* | false |
392395

393396

394397
## F.A.Q.

‎documentation/promise-api.md

Lines changed: 10 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -52,37 +52,23 @@ const mariadb = require('mariadb');
5252

5353
## Migrating from 2.x or mysql/mysql2 to 3.x
5454

55-
This is a breaking change from 3.0 compare to previous version and mysql/mysql2 drivers.
55+
Default behaviour for decoding [BIGINT](https://mariadb.com/kb/en/bigint/) / [DECIMAL](https://mariadb.com/kb/en/decimal/) datatype for 2.x version and mysql/mysql2 drivers return a javascript [Number](https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Global_Objects/Number) object.
56+
BIGINT/DECIMAL values might not be in the safe range, resulting in approximate results.
5657

57-
Integers in JavaScript use IEEE-754 representation. This means that Node.js cannot exactly represent integers in the ±9,007,199,254,740,991 range.
58-
However, MariaDB does support larger integers and exact big decimal.
59-
This means that when the value set on a BIGINT is not in the [safe](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isSafeInteger) range, javascript Number Object cannot represent exact value.
60-
Same with DECIMAL type that might not be exact in IEEE 754 floating number.
61-
62-
2.x/mysql/mysql2 have options that try to address those issues. Those are now removed :
63-
64-
* supportBigNumbers: When an integer was not in the safe range, the driver did interprets the value as a [`Long`](https://www.npmjs.com/package/long) object. Now use javascript standard BigInt
65-
* supportBigInt: Removed, since BigInt support is now required.
66-
* bigNumberStrings: When an integer was not in the safe range, the driver did interprets the value as a string.
67-
68-
Those options might return a different object type depending on value.
69-
70-
71-
Since 3.x version, Driver automatically return data depending on data types :
58+
Since 3.x version, driver has reliable default, returning:
7259
* DECIMAL => javascript String
73-
* BIGINT => javascript BigInt
60+
* BIGINT => javascript [BigInt](https://mariadb.com/kb/en/bigint/) object
7461

75-
this permit to ensure returning exact values and reliable data type, but might cause some incompatibility.
76-
Driver provides 3 options to address this issue.
62+
For compatibility with previous version or mysql/mysql driver, 3 options have been added to return BIGINT/DECIMAL as number, as previous defaults.
7763

7864
|option|description|type|default|
7965
|---:|---|:---:|:---:|
8066
| **insertIdAsNumber** | Whether the query should return last insert id from INSERT/UPDATE command as BigInt or Number. default return BigInt |*boolean* | false |
81-
| **decimalAsNumber** | Whether the query should return decimal as Number. If enable, this might return approximate values. |*boolean* | false |
82-
| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enable, this might return approximate values. |*boolean* | false |
67+
| **decimalAsNumber** | Whether the query should return decimal as Number. If enabled, this might return approximate values. |*boolean* | false |
68+
| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enabled, this might return approximate values. |*boolean* | false |
69+
70+
Previous options `supportBigNumbers` and `bigNumberStrings` still exist for compatibility, but are now deprecated.
8371

84-
If wanting compatibility with previous version those values can be set to true / use [`typeCast`](#typeCast) to convert DECIMAL/BIGINT to expect value.
85-
8672

8773
## Recommendation
8874

@@ -552,6 +538,7 @@ connection.query('select * from animals')
552538
* [`rowsAsArray`](#rowsAsArray)
553539
* [`nestTables`](#nestTables)
554540
* [`dateStrings`](#dateStrings)
541+
* [`bigIntAsNumber`](#bigIntAsNumber)
555542
* [`decimalAsNumber`](#decimalAsNumber)
556543

557544
Those options can be set on the query level, but are usually set at the connection level, and will then apply to all queries.

‎lib/cmd/command.js

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ class Command extends EventEmitter {
8888
if (this.reject) {
8989
if (this.stack) {
9090
err = Errors.createError(
91-
err.message,
91+
err.text ? err.text : err.message,
9292
err.sql,
9393
err.fatal,
9494
info,
@@ -126,7 +126,14 @@ class Command extends EventEmitter {
126126

127127
const affectedRows = packet.readUnsignedLength();
128128
let insertId = packet.readSignedLengthBigInt();
129-
if (opts.insertIdAsNumber && insertId != null) insertId = Number(insertId);
129+
if (insertId != null && (opts.supportBigNumbers || opts.insertIdAsNumber)) {
130+
if (
131+
opts.supportBigNumbers &&
132+
(opts.bigNumberStrings || !Number.isSafeInteger(Number(insertId)))
133+
) {
134+
insertId = insertId.toString();
135+
} else insertId = Number(insertId);
136+
}
130137
info.status = packet.readUInt16();
131138

132139
const okPacket = new OkPacket(affectedRows, insertId, packet.readUInt16());

‎lib/cmd/decoder/binary-decoder.js

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,14 +60,26 @@ class BinaryDecoder {
6060
return packet.readDouble();
6161
case FieldType.BIGINT:
6262
const val = column.signed() ? packet.readBigInt64() : packet.readBigUInt64();
63-
if (opts.bigIntAsNumber && val != null) {
63+
if (val != null && (opts.bigIntAsNumber || opts.supportBigNumbers)) {
64+
if (
65+
opts.supportBigNumbers &&
66+
(opts.bigNumberStrings || !Number.isSafeInteger(Number(val)))
67+
) {
68+
return val.toString();
69+
}
6470
return Number(val);
6571
}
6672
return val;
6773
case FieldType.DECIMAL:
6874
case FieldType.NEWDECIMAL:
6975
const valDec = packet.readDecimalLengthEncoded();
70-
if (opts.decimalAsNumber && valDec != null) {
76+
if (valDec != null && (opts.decimalAsNumber || opts.supportBigNumbers)) {
77+
if (
78+
opts.supportBigNumbers &&
79+
(opts.bigNumberStrings || !Number.isSafeInteger(Number(valDec)))
80+
) {
81+
return valDec.toString();
82+
}
7183
return Number(valDec);
7284
}
7385
return valDec;

‎lib/cmd/decoder/text-decoder.js

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,14 +31,26 @@ class TextDecoder {
3131
return packet.readFloatLengthCoded();
3232
case FieldType.BIGINT:
3333
const val = packet.readBigIntLengthEncoded();
34-
if (opts.bigIntAsNumber && val != null) {
34+
if (val != null && (opts.bigIntAsNumber || opts.supportBigNumbers)) {
35+
if (
36+
opts.supportBigNumbers &&
37+
(opts.bigNumberStrings || !Number.isSafeInteger(Number(val)))
38+
) {
39+
return val.toString();
40+
}
3541
return Number(val);
3642
}
3743
return val;
3844
case FieldType.DECIMAL:
3945
case FieldType.NEWDECIMAL:
4046
const valDec = packet.readDecimalLengthEncoded();
41-
if (opts.decimalAsNumber && valDec != null) {
47+
if (valDec != null && (opts.decimalAsNumber || opts.supportBigNumbers)) {
48+
if (
49+
opts.supportBigNumbers &&
50+
(opts.bigNumberStrings || !Number.isSafeInteger(Number(valDec)))
51+
) {
52+
return valDec.toString();
53+
}
4254
return Number(valDec);
4355
}
4456
return valDec;

‎lib/cmd/parser.js

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,17 @@ class Parser extends Command {
132132
bigIntAsNumber:
133133
cmdOpts.bigIntAsNumber != undefined ? cmdOpts.bigIntAsNumber : connOpts.bigIntAsNumber,
134134
decimalAsNumber:
135-
cmdOpts.decimalAsNumber != undefined ? cmdOpts.decimalAsNumber : connOpts.decimalAsNumber
135+
cmdOpts.decimalAsNumber != undefined ? cmdOpts.decimalAsNumber : connOpts.decimalAsNumber,
136+
insertIdAsNumber:
137+
cmdOpts.insertIdAsNumber != undefined
138+
? cmdOpts.insertIdAsNumber
139+
: connOpts.insertIdAsNumber,
140+
supportBigNumbers:
141+
cmdOpts.supportBigNumbers != undefined
142+
? cmdOpts.supportBigNumbers
143+
: connOpts.supportBigNumbers,
144+
bigNumberStrings:
145+
cmdOpts.bigNumberStrings != undefined ? cmdOpts.bigNumberStrings : connOpts.bigNumberStrings
136146
};
137147
}
138148

@@ -147,7 +157,7 @@ class Parser extends Command {
147157
* @returns {*} null or {Resultset.readResponsePacket} in case of multi-result-set
148158
*/
149159
readOKPacket(packet, out, opts, info) {
150-
const okPacket = Command.parseOkPacket(packet, out, opts, info);
160+
const okPacket = Command.parseOkPacket(packet, out, this.opts, info);
151161
this._rows.push(okPacket);
152162

153163
if (info.status & ServerStatus.MORE_RESULTS_EXISTS) {

‎lib/config/connection-options.js

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,16 +145,21 @@ class ConnectionOptions {
145145
}
146146
this.permitLocalInfile = this.pipelining ? false : opts.permitLocalInfile || false;
147147
}
148-
this.insertIdAsNumber = opts.insertIdAsNumber || false;
149-
this.decimalAsNumber = opts.decimalAsNumber || false;
150-
this.bigIntAsNumber = opts.bigIntAsNumber || false;
151148
this.prepareCacheLength = opts.prepareCacheLength === undefined ? 256 : opts.prepareCacheLength;
152149
this.restrictedAuth = opts.restrictedAuth;
153150
if (this.restrictedAuth !== undefined && this.restrictedAuth !== null) {
154151
if (!Array.isArray(this.restrictedAuth)) {
155152
this.restrictedAuth = this.restrictedAuth.split(',');
156153
}
157154
}
155+
156+
// for compatibility with 2.x version and mysql/mysql2
157+
this.bigIntAsNumber = opts.bigIntAsNumber || false;
158+
this.insertIdAsNumber = opts.insertIdAsNumber || false;
159+
this.decimalAsNumber = opts.decimalAsNumber || false;
160+
this.supportBigNumbers = opts.supportBigNumbers || false;
161+
this.bigNumberStrings = opts.bigNumberStrings || false;
162+
158163
if (this.maxAllowedPacket && !Number.isInteger(this.maxAllowedPacket)) {
159164
throw new RangeError(
160165
"maxAllowedPacket must be an integer. was '" + this.maxAllowedPacket + "'"

0 commit comments

Comments
 (0)