Open
Description
Specifications
- Client Version: 1.40.0
- InfluxDB Version: Cloud 2.0 TSM
- Platform: GCP
I am extracting some data from BQ and pushing it to Influx. That part is working well.
Next, I write a point in an Influx bucket to mark the task completed. This was written by someone else and it uses a different call.
In this particular case, I have a conflict for the second parameter. In the GCP logs, I can see the error as "default" on top of the log generated by loguru (the log appears twice).
Code sample to reproduce problem
in the calling function
points = [
Point.from_dict(
{
"measurement": __DATA_MEASUREMENT,
"fields": {state: value},
"tags": {"type": "status", **tags},
}
),
Point.from_dict(
{
"measurement": __DATA_MEASUREMENT,
"fields": {state: datetime.now().isoformat()},
"tags": {"type": "timestamp", **tags},
}
),
]
write_points(
__INFLUX_URL,
__INFLUX_TOKEN,
__INFLUX_BUCKET,
__INFLUX_ORGANISATION,
points,
)
```
in the lib to write_points:
``````python
def write_points(
influx_url: str,
influx_token: str,
influx_bucket: str,
influx_organization: str,
points: List[Union[Point, Dict[str, Any]]],
) -> None:
logger.info("Writing to InfluxDB, Points length={}", len(points))
client = InfluxDBClient(url=influx_url, token=influx_token, timeout=300_000)
try:
write_api = client.write_api(
success_callback=callback.success,
error_callback=callback.error,
retry_callback=callback.retry,
write_options=WriteOptions(max_retries=10, max_retry_time=280_000)
)
try:
write_api.write(
batch_size=200,
bucket=influx_bucket,
org=influx_organization,
record=points
)
logger.debug("Write operation completed")
except InfluxDBError as e:
logger.error("InfluxDB Asynchronous write error: {}", str(e))
if hasattr(e, 'response'):
logger.error("Response status: {}", e.response.status_code)
logger.error("Response headers: {}", dict(e.response.headers))
logger.error("Response body: {}", e.response.text)
raise
except Exception as ex:
logger.error("Unexpected error during write operation: {}", str(ex))
raise
finally:
client.close()
```
I know this is not correctly configured to do SYNCHRONOUS write, but that is how I see the issue.
The callback object is just implementing calls to logger.{info|error|warning} resp.
### Expected behavior
The logs are formatted as per my loguru logger settings
### Actual behavior
some logs (error 400 schema conflict) appear twice.
### Additional info
(DEBUG) 2025-04-25 15:15:06.130 CEST
Write operation completed
(DEFAULT) 2025-04-25 15:15:10.914 CEST
The batch item wasn't processed successfully because: (400)
(DEFAULT) 2025-04-25 15:15:10.914 CEST
Reason: Bad Request
(DEFAULT) 2025-04-25 15:15:10.914 CEST
HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 25 Apr 2025 13:15:09 GMT', 'Content-Type': 'application/json', 'Content-Length': '247', 'Connection': 'keep-alive', 'influxdata-partial-write': 'true', 'trace-id': 'xxxxxxxxxxxxxxxx', 'trace-sampled': 'false', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Influxdb-Request-ID': 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz', 'X-Influxdb-Build': 'Cloud'})
(DEFAULT) 2025-04-25 15:15:10.914 CEST
HTTP response body: {"code":"invalid","message":"partial write has occurred, errors encountered on line(s): line 2: batch schema conflict - line has column 'inserted_data' of type iox::column_type::field::string but type is iox::column_type::field::integer","line":2}
(ERROR) 2025-04-25 15:15:10.915 CEST
Cannot write batch: ('{bucket_name}', '{org_name}', 'ns'), data: b'GCP_Data_monitoring,app=myapp,stage=preprod,type=status,zone=eu-west-01 inserted_data=1i\nGCP_Data_monitoring,app=myapp,stage=preprod,type=timestamp,zone=eu-west-01 inserted_data="2025-04-25T13:15:06.127660"' due: (400) Reason: Bad Request HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 25 Apr 2025 13:15:09 GMT', 'Content-Type': 'application/json', 'Content-Length': '247', 'Connection': 'keep-alive', 'influxdata-partial-write': 'true', 'trace-id': 'xxxxxxxxxxxxxxxx', 'trace-sampled': 'false', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Influxdb-Request-ID': 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz', 'X-Influxdb-Build': 'Cloud'}) HTTP response body: {"code":"invalid","message":"partial write has occurred, errors encountered on line(s): line 2: batch schema conflict - line has column 'inserted_data' of type iox::column_type::field::string but type is iox::column_type::field::integer","line":2}