There are a few things to consider:
- If there are string variables in the datafile, it takes RB much longer to iterate through the dataset. The SPSS file format has a horrible way of storing strings.
- The hard drive speed is usually the bottleneck, so placing the file on a network drive will kill performance. Putting the file even temporarily on the local hard drive is recommended.
- The number of variables and records affect speed of computations – the larger the file, the longer it takes RB to iterate through it.
- There’s theoretically no limit to file size.
- It’s usually more efficient to create multiple smaller tables. Larger tables require more RAM to process and could take longer than breaking it out.