Simplify csv reader/writer

The stumbling stone for many users is to create the proper SQL table structure before CSV data is loaded.

If the CSV file contains a header, then the database table creation can be based on a simple analysis of the header line and a sample of the remaining rows. Consider a simple wrapper around the SQL CREATE + COPY (INTO/TO) action.

monetdbe.read_csv(, sep=',', quotechar='"', null='')
The parallel MonetDB CSV reader should make a difference compared to plain Python and Pandas

It could also be aligned with the pandas.read_csv() interface
data = pd.read_csv(
"data/files/complex_data_example.tsv", # relative python path to subdirectory
sep='\t' # Tab-separated value file.
quotechar="'", # single quote allowed as quote character
dtype={"salary": int}, # Parse the salary column as an integer
usecols=['name', 'birth_date', 'salary']. # Only load the three columns specified.
parse_dates=['birth_date'], # Intepret the birth_date column as a date
skiprows=10, # Skip the first 10 rows of the file
na_values=['.', '??'] # Take any '.' or '??' values as NA
)

The complimentary action is monetdb.write_csv()

[gijzelaerr]

Nice, this is quite easy to do, we can just use the read_csv() from pandas and then import the resulting DataFrame.

Well, I am not sure. I guess that the MonetDB csv reader is much faster, because it exploits a lot of parallelism and directly builds the bats. First parsing it using pandas might be a lot slower.
The wrapper probably attempts to read the meta data, which could be done with a pandas.read_csv of the first N lines and then delegates the bulk directly to MonetDB

[gijzelaerr]

then I propose to add the Pandas method to 0.9 since it is easy to implement. Then for a future release, we can add the MonetDB accelerated version.

I really want to get 0.9 out of the door tomorrow, so I'm going to be strict on what is going to land in this release. The Windows build probably need to most work.

agree?

Agreed.

[gijzelaerr]

implemented (using pandas)