Sometimes we need to use flat files to load data from third party databases into BW. BW can load data from two types of flat files, CSV and ASCII Files (Strictly speaking CSV is particual type of ASCII file, but from BW perspective let us treat CSV and ASCII as two different formats). The performance is not an issue for small files. However, load performance becomes critical when dealing with large files with hundres of thousands or millions of rows.
In general we use CSV files because they are easy to generate and maintain. But by using ASCII file instead of a CSV file we can improve the performance by 30% to 40%. However, generating ASCII files is not as straight forward as generating ASCII files. In CSV files each field is sperated by a delemeter (in general comma) and each field is also enclosed by optional text qualifiers (commonly double quotes). CSV file does not care about length of each field.
Sample CSV Format
Sales Org Distribution Channel Amount
"1000","2","800"
"1002","3","950"
There is no delimter or text qualifier in ASCII file. All fields are stored as a continuous string for each record and records are seperated by Carriage Return
Sample ASCII Format
Sales Org Distribution Channel Amount
0000001002002800
0000001003003950
As one can see from the examples above in case of CSV it is for BW to know which value is for which field as there is a delemeter but that is not the case iwith ASCII file. For ASCII files BW relies on length of the field to know which value is for which field. For above example it knows that Sales Org is of length 10 so takes first 10 characters (0000001000) in the records assigns to sales org, Distribution Channel is of length 3 so assigns next 3 characters (002) to it and so on.
As you can see, it is very important to append leading zeros or spaces to field values in case of ASCII file otherwise incorrect values will be assigned to fields. Because of this generating ASCII files is some what complicated than generating CSV files.
Loading ASCII files into BW is much faster as it does not look for delemters or escape characters. How fast? 30 to 40% depending on file size, record length and othe parameters.
You can select the type of the file you are loading under "External Data Tab" of an infopackage as shown in below screenshot.
