BUG: read_csv with engine=pyarrow and numpy-nullable dtype

[jbrockmendel]

• closes BUG: read_csv loses precision when engine='pyarrow' and dtype Int64 #56136 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Also makes this code path robust to always-distinguish behavior in #62040

[mroeschke]

From the original issue, do you know where we are introducing float to lose precision when wanting the result type to be int?

[jbrockmendel]

From the original issue, do you know where we are introducing float to lose precision when wanting the result type to be int?

In arrow_table_to_pandas the pyarrow[int64] columns get converted to np.float64, then in finalize_pandas_output that gets cast back to Int64.

[mroeschke]

OK I see, it's pyarrow.Table.to_pandas casting the int to float when there's null.

What if in arrow_table_to_pandas, we always provide fallback type_mapper={pyarrow ints : pandas nullable ints} to avoid the lossy conversions, then afterwards we cast the pandas nullable ints to the appropriate type?

[jbrockmendel]

That’s basically what this is currently doing, just not in that function since it is also called from other places.

I’m out of town for a few days. If you feel strongly that this logic should live inside that function I’ll move it when I get back