Can't load large (1gb) stata file

[bdemeshev]

I am trying to read stata panel datasets of Russian households and invividuals.
They can be downloaded freely after simple registration.
The houeseholds dataset is ~1gb and individuals is ~4gb. Machine has 16 gb.

Even 1gb file take too long to load (have waited > 15 minutes without any results).

using StatFiles
a = load("USER_RLMS-HSE_HH_1994_2017nt_eng.dta")

[VDoetsch]

I have a similar issue with a stata panel dateset which is ~ 50 mb. It is loading for more than an hour but nothing happens.

[nalimilan]

I have the same problem with a Sata dataset with more than 1000 columns (but only about 1000 rows). ReadStatTables.jl is able to read it almost instantly, so I suspect it's due to StatFiles iterating row-by-row using a 1000-element NamedTuple, which is very demanding for the compiler.