BUG: read_csv() with engine="pyarrow" converts numeric string even when dtype=str is specified #58260
Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
demo_df: pd.DataFrame = pd.DataFrame(
{
"sample_name": ["sample_1", "sample_2"],
"case_id": ["00001", "00002"],
"sample_id": ["00001001", "00001002"],
"sample_type": ["T", "T"],
}
)
demo_df.to_csv("bug_demo.csv", header=True, index=False, sep=",", encoding="utf-8")
pyarrow_df: pd.DataFrame = pd.read_csv(
"bug_demo.csv",
header=0,
index_col=None,
sep=",",
encoding="utf-8",
engine="pyarrow",
dtype=str,
)
print("This is read by pyarrow engine.")
print(pyarrow_df)
c_df: pd.DataFrame = pd.read_csv(
"bug_demo.csv",
header=0,
index_col=None,
sep=",",
encoding="utf-8",
engine="c",
dtype=str,
)
print("This is read by c engine.")
print(c_df)
python_df: pd.DataFrame = pd.read_csv(
"bug_demo.csv",
header=0,
index_col=None,
sep=",",
encoding="utf-8",
engine="python",
dtype=str,
)
print("This is read by python engine.")
print(python_df)Issue Description
I have encountered an issue with the read_csv() function in pandas when using the pyarrow engine. Even when specifying dtype=str, pure numeric strings are being converted to numeric type. Additionally, pure numeric strings starting with multiple zeros lose the leading zeros in the resulting DataFrame. This behavior is unexpected as I would like to preserve the original format of the numeric strings as text.
Expected Behavior
The example demonstrates reading a CSV file with different engines. It is expected that pyarrow engine should get the same DataFrame as c engine and python engine when using dtype=str. It should output the following texts.
# This is read by pyarrow engine.
sample_name case_id sample_id sample_type
0 sample_1 1 1001 T
1 sample_2 2 1002 T
# This is read by c engine.
sample_name case_id sample_id sample_type
0 sample_1 00001 00001001 T
1 sample_2 00002 00001002 T
# This is read by python engine.
sample_name case_id sample_id sample_type
0 sample_1 00001 00001001 T
1 sample_2 00002 00001002 T
Installed Versions
INSTALLED VERSIONS
commit : d9cdd2e
python : 3.12.2.final.0
python-bits : 64
OS : Linux
OS-release : 6.5.0-26-generic
Version : #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 12 10:22:43 UTC 2
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : None
LOCALE : en_US.UTF-8
pandas : 2.2.2
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.9.0
setuptools : 69.5.1
pip : 24.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 5.1.0
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.3
IPython : 8.22.2
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
bottleneck : 1.3.8
dataframe-api-compat : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : 2.9.0
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 11.0.0
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.12.0
sqlalchemy : None
tables : None
tabulate : 0.9.0
xarray : None
xlrd : None
zstandard : 0.22.0
tzdata : 2024.1
qtpy : None
pyqt5 : None