,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
SAP BW Business Content: valued, but not a wishing well...
As we can read from help.sap.com, SAP Business Information Warehouse provides pre-configured objects under the collective term "Business Content" (BC). These objects accelerate the implementation of SAP BW, since they deliver ready solutions, meeting the requirements for business information.
As regards our analysis area (BW extraction tools), it’s evident that, coming out from SAP official definition and just working for some period on the system, BC (and its datasources) mean ready to run build-in extractors, a good (and growing) business coverage within SAP environments (BW API Service is available as plug-in for all R/3 systems, BW itself and therefore also in APO, mySAP ERP components and industry solutions), transactional and master data involved, less implementation efforts and costs and sophisticated delta handling.
And you will say "This is a dreamland !".
I’m sorry, my dear friend, I hate to be a killjoy, but, wake up and welcome in the reality...
Although the BC and the related extraction technology have reached a significant coverage of every business area, there are still a lot of reasons (or, better, there are a lot of situations in which you are compelled) to enhance existing extractors or even develop entirely custom extractors: need to extract customer-specific data, to build customer extension for the BC that doesn’t support specific information required by the customer and so on.
Besides, we can’t forget about modified SAP systems in some areas with specific customizing settings or simply with custom fields added to standard tables.
Now the spontaneous question is: what’s happen if a standard logistic datasource as provided in standard (ready-to-use) configuration doesn’t completely meet our data model requirements ?
In other words, we go in RSA5 transaction screen (Installation of datasource from Business Content), by surfing through application component hierarchy, we find our candidate datasource 2LIS_11_VASCL (since functional analysis requires a set of information belonging to sales order schedule line level, for example); afterwards, a double-click on it and we inspect the field list and - dash it ! - I don’t see a specific needed field (e.g., AUDAT, the document date) !
What I have to do ?
As regards our analysis area (BW extraction tools), it’s evident that, coming out from SAP official definition and just working for some period on the system, BC (and its datasources) mean ready to run build-in extractors, a good (and growing) business coverage within SAP environments (BW API Service is available as plug-in for all R/3 systems, BW itself and therefore also in APO, mySAP ERP components and industry solutions), transactional and master data involved, less implementation efforts and costs and sophisticated delta handling.
And you will say "This is a dreamland !".
I’m sorry, my dear friend, I hate to be a killjoy, but, wake up and welcome in the reality...
Although the BC and the related extraction technology have reached a significant coverage of every business area, there are still a lot of reasons (or, better, there are a lot of situations in which you are compelled) to enhance existing extractors or even develop entirely custom extractors: need to extract customer-specific data, to build customer extension for the BC that doesn’t support specific information required by the customer and so on.
Besides, we can’t forget about modified SAP systems in some areas with specific customizing settings or simply with custom fields added to standard tables.
Now the spontaneous question is: what’s happen if a standard logistic datasource as provided in standard (ready-to-use) configuration doesn’t completely meet our data model requirements ?
In other words, we go in RSA5 transaction screen (Installation of datasource from Business Content), by surfing through application component hierarchy, we find our candidate datasource 2LIS_11_VASCL (since functional analysis requires a set of information belonging to sales order schedule line level, for example); afterwards, a double-click on it and we inspect the field list and - dash it ! - I don’t see a specific needed field (e.g., AUDAT, the document date) !
What I have to do ?
Extraction cockpit technique: let's go back a little...
Fig.1: LC Delta Process for Sales Order Schedule Lines
With the LC, several data structures are delivered and, for each level of detail, there exists an extract structure as well as a datasource (that already represents a BW extract view).
When you create and save a sales order (as other transactional tasks), the document is processed in the memory and then stored into application (and database) tables.
In LC extraction technique (see Fig.1) we have at our disposal different LIS communications structures (like the MCVBAK, MCVAP, MCVEP and so on for sales orders) that we can decide to use for our reporting purposes when the application is running and during memory processing (into a separate memory partition, but for details refer to LOGISTIC COCKPIT DELTA MECHANISM Weblog Series).
To be more precise, every extract structure is related to one or more communication structures (and for every communication structure involved an include is provided by standard; see Fig.2) : for sales order schedule line extract structure we have got MCVBAK, MCVAP, MCVEP, MCVBKD, MCVBUK and MCVBUP, whose components you can see from SE11).
Fig.2: 2LIS_11_VASCL Include mapping
In search of lost field (as Proust said...)
Now let’s come back to our little example and try to understand what procedure have to be followed when, as in our situation, we need a field not provided in a ready-to-run configuration.
2LIS_11_VASCL is the standard LC datasource to extract order schedule lines related information. MC11VA0SCL represents its linked extract structure.
Remember that it’s possible to enhance that, but you can’t create new extract structures (on the same standard datasource).
In the LC there also events mentioned (shown below the extraction structure), but they don’t have any customizing options. They are here just to give you some kind to transparency when our structure will be updated, but they are not of any relevance from the customizing point of view.
2LIS_11_VASCL is the standard LC datasource to extract order schedule lines related information. MC11VA0SCL represents its linked extract structure.
Remember that it’s possible to enhance that, but you can’t create new extract structures (on the same standard datasource).
In the LC there also events mentioned (shown below the extraction structure), but they don’t have any customizing options. They are here just to give you some kind to transparency when our structure will be updated, but they are not of any relevance from the customizing point of view.
Among existing fields from the available communication structures
Within LC (see Fig.3) a tool is provided that enables you to add fields from the LIS communication structures (to the extract structure) without having to do any modifications.
Fig.3: Logistic Cockpit Customizing screen
In the maintenance screen (see Fig.4), on the left side, you see what has already been selected in the standard extract structure and on the right side, you see all the available fields of the communication structures where you can select fields from for the update.
Fig.4: Maintenance screen
And, what my eyes are seeing at a glance ? My AUDAT field !
Ok, now it’s enough to highlight the row and click on the left-arrow: (every) selected field is included automatically in a generated append structure for the corresponding include structure of the extract structure (for example, append ZZMC11VA1SCL for include MC11VA1SCL for additional fields in the order schedule lines extractor for LIS communication structure MCVBAK).
When you successfully complete this step, the traffic light icon turns red. This indicates that you changed the structure.
At this point, you have to generate the datasource (see Fig.5): here you can (among the other things) choose fields that can be selected (for various reasons, it is not possible to offer all the fields contained in the LIS communication structure for selection in the extract structure; these fields are hidden for a specific purpose because some specific extract structure is a combination of different processes; for details see OSS Note 351214 ‘BW extraction SD: Restricted field selection’) and if a key figure is inverted or not (refer to OSS Note 382779 ‘Cancellation field in the datasource maintenance’ for details).
After maintenance in this step, the traffic light turns yellow.
Fig.5: Datasource generation
Once you activate the update, data is written to the extract structure and the traffic light then turns green. Our enhancement process is completed and now you can schedule (if required by your delta method) the delta job control process.
If, during a subsequent import of a new plug-in, this same field is already included in the standard extract structure, it will be removed from the customer enhancement (in order to avoid a double occurrence for the same field) thanks to an automatic XPRA program execution within the upgrade procedure.
When you extend the extraction structures in the LC, you can realize that not all existing fields of an assigned LIS communication structure are available for selection.
This is not a lapse of memory: this behavior is wanted.
As not all fields of the communication structures can be used in a practical way, some are hidden because, for example, the field is not filled for the relevant events or is only used internally or for other reasons of design (e.g. you should select key figures only from the most detailed communication structure and only the characteristics from all communication structures !).
Fig.3: Logistic Cockpit Customizing screen
In the maintenance screen (see Fig.4), on the left side, you see what has already been selected in the standard extract structure and on the right side, you see all the available fields of the communication structures where you can select fields from for the update.
Fig.4: Maintenance screen
Ok, now it’s enough to highlight the row and click on the left-arrow: (every) selected field is included automatically in a generated append structure for the corresponding include structure of the extract structure (for example, append ZZMC11VA1SCL for include MC11VA1SCL for additional fields in the order schedule lines extractor for LIS communication structure MCVBAK).
When you successfully complete this step, the traffic light icon turns red. This indicates that you changed the structure.
At this point, you have to generate the datasource (see Fig.5): here you can (among the other things) choose fields that can be selected (for various reasons, it is not possible to offer all the fields contained in the LIS communication structure for selection in the extract structure; these fields are hidden for a specific purpose because some specific extract structure is a combination of different processes; for details see OSS Note 351214 ‘BW extraction SD: Restricted field selection’) and if a key figure is inverted or not (refer to OSS Note 382779 ‘Cancellation field in the datasource maintenance’ for details).
After maintenance in this step, the traffic light turns yellow.
Fig.5: Datasource generation
Once you activate the update, data is written to the extract structure and the traffic light then turns green. Our enhancement process is completed and now you can schedule (if required by your delta method) the delta job control process.
If, during a subsequent import of a new plug-in, this same field is already included in the standard extract structure, it will be removed from the customer enhancement (in order to avoid a double occurrence for the same field) thanks to an automatic XPRA program execution within the upgrade procedure.
When you extend the extraction structures in the LC, you can realize that not all existing fields of an assigned LIS communication structure are available for selection.
This is not a lapse of memory: this behavior is wanted.
As not all fields of the communication structures can be used in a practical way, some are hidden because, for example, the field is not filled for the relevant events or is only used internally or for other reasons of design (e.g. you should select key figures only from the most detailed communication structure and only the characteristics from all communication structures !).
Enhance it, but mind the queue !
If you change an extract structure in the Logistic Cockpit through transaction LBWE (or one of the LIS communication structures which are the basis for the extract structure or, in individual cases, also an application table - e.g. MSEG, it’s happened to me ! - by importing a transport request or a support package or by carrying out an upgrade, you have to operate with a lot of cautiousness.
Many problems in this area result from the fact that, although everything is well organized in the development system, the transport that takes place in the production system is not controlled.
In fact, a not responsible and diligent behaviour (when your datasource had already been activated for update, even if for a very short period) can lead to various errors: delta requests terminate, the update from the extraction queue does not finish or V3 update is no longer processed, the initialization on data that was generated before the change no longer works, the protocol terminates...in short, a real tragedy !
Without venturing on a too technical ground, this situation can be briefly described in this way: when you change a structure, the data which is stored in the old form can no longer be interpreted correctly by the new version of the same extract structure.
For the same reason, you can no longer use the statistical data already contained in setup tables and you have to delete it via transaction LBWG.
Therefore, you should carry out the following steps before you change the extract structure (also for an R/3 release upgrade or plug-in/support package import):
Many problems in this area result from the fact that, although everything is well organized in the development system, the transport that takes place in the production system is not controlled.
In fact, a not responsible and diligent behaviour (when your datasource had already been activated for update, even if for a very short period) can lead to various errors: delta requests terminate, the update from the extraction queue does not finish or V3 update is no longer processed, the initialization on data that was generated before the change no longer works, the protocol terminates...in short, a real tragedy !
Without venturing on a too technical ground, this situation can be briefly described in this way: when you change a structure, the data which is stored in the old form can no longer be interpreted correctly by the new version of the same extract structure.
For the same reason, you can no longer use the statistical data already contained in setup tables and you have to delete it via transaction LBWG.
Therefore, you should carry out the following steps before you change the extract structure (also for an R/3 release upgrade or plug-in/support package import):
close your system and make sure that no updates are performed (both by users or batch/background process);
start the update collective run directly from LC (that concerns either the V3 update or the Delta Queued update);At this moment, with "delta direct" method, the delta queue (RSA7) must be empty, with "delta queued" method, the extraction queue (LBWQ) and delta queue (RSA7) must be empty, with "unserialized V3 update", the extraction queue (SM13) and the delta queue (RSA7) must be empty.
load all data of the respective datasources into your BW System
. To completely empty the delta queue, request a delta twice, one after the other: in this way the second upload will transfer 0 data records, but only with the second upload is the data of the delta queue that is available as a delta repeat deleted.
Anyway, with plug-in PI 2000.2 (or PI-A 2000.2) specific checks were implemented in LBWE so that structures are changed only if (in this order) there are no entries in setup tables of the affected application, there are no entries in the V3 update and in the queue for the affected application. Now, that all data containers of the relevant data flow are empty, you can make (import) the change.
IMPORTANT: the fields that are available by default in LBWE are automatically filled during the extraction and are delta relevant(see later in this weblog for more details about ‘delta relevant’ changes).
. To completely empty the delta queue, request a delta twice, one after the other: in this way the second upload will transfer 0 data records, but only with the second upload is the data of the delta queue that is available as a delta repeat deleted.
Anyway, with plug-in PI 2000.2 (or PI-A 2000.2) specific checks were implemented in LBWE so that structures are changed only if (in this order) there are no entries in setup tables of the affected application, there are no entries in the V3 update and in the queue for the affected application. Now, that all data containers of the relevant data flow are empty, you can make (import) the change.
IMPORTANT: the fields that are available by default in LBWE are automatically filled during the extraction and are delta relevant(see later in this weblog for more details about ‘delta relevant’ changes).
Using the LIS enhancement on available communication structures
If your field is not available in LC (that is, it is not in the available communication structures) you have to follow some different ways.
One method of adding user-defined fields is the following: add the required fields to the communication structures (MCVBAK, MCVBAP and so on ) using the append method (via SE11) and then use the LIS customer exits to fill the field.
For information on enhancing the communication structures, you can see the documentation for the enhancements MCS10001, MCS50001 and MCS60001 provided in transaction SMOD.
After you enhance the communication structures you can then enhance the extract structure with the relevant field in the customizing cockpit (transaction LBWE), provided that the communication structure is available in the selection. Then you can proceed with the steps described before in the previous bullet.
Even this procedure allows to manage delta records, but you must make sure that you can determine the status in the user exit before and after the document change (this varies from every peculiar situation and from which table the field is filled, e.g. internal document tables, such as XVBAP and YVBAP).
Why this is so important ?
A document change in the delta extraction process (relating to our LC datasources) consists in the transfer of two data records to BW: one of these records represents the status of the document before the change (before image) and the other one represents the status after the change (after image).
During the extraction process, these two data records are compared by the system and checked for changes: only if there is some difference between the before and after images, these records will be involved in the extraction process and will continue to be processed.
Please refer to OSS Notes 216448 ‘BW/SIS: Incorrect update / SD user exit‘ and 757361 ‘Additional data records in BW when document changed’ for more information on correctly populating the before and after image (even if related to only SD Applications).
One method of adding user-defined fields is the following: add the required fields to the communication structures (MCVBAK, MCVBAP and so on ) using the append method (via SE11) and then use the LIS customer exits to fill the field.
For information on enhancing the communication structures, you can see the documentation for the enhancements MCS10001, MCS50001 and MCS60001 provided in transaction SMOD.
After you enhance the communication structures you can then enhance the extract structure with the relevant field in the customizing cockpit (transaction LBWE), provided that the communication structure is available in the selection. Then you can proceed with the steps described before in the previous bullet.
Even this procedure allows to manage delta records, but you must make sure that you can determine the status in the user exit before and after the document change (this varies from every peculiar situation and from which table the field is filled, e.g. internal document tables, such as XVBAP and YVBAP).
Why this is so important ?
A document change in the delta extraction process (relating to our LC datasources) consists in the transfer of two data records to BW: one of these records represents the status of the document before the change (before image) and the other one represents the status after the change (after image).
During the extraction process, these two data records are compared by the system and checked for changes: only if there is some difference between the before and after images, these records will be involved in the extraction process and will continue to be processed.
Please refer to OSS Notes 216448 ‘BW/SIS: Incorrect update / SD user exit‘ and 757361 ‘Additional data records in BW when document changed’ for more information on correctly populating the before and after image (even if related to only SD Applications).
Using custom append on the extract structure
If you don’t find your field already available within LBWE, if, for any reason, you don’t want (or you are not able) to enhance LIS communication structures, you have another chance: enhance your extract structures by creating an append with your ZZ* fields and then filling these fields with a specific user-exit.
To do this, go to RSA6, choose your datasource, double-click on it and then on the extract structure: you will see an SE16 screen, create an append, insert your ZZ* fields, save.
Then you have to fill those fields with some ABAP custom code that can be anything from some simple calculations or table lookups to complex business logic requiring access to multiple database tables. You can do that by CMOD, creating a project and using the enhancement RSAP0001.
The function modules provided in this enhancement serve for the derivation or modification of data, that is extracted and transferred by the extraction engine of the Business Information Warehouse: EXIT_SAPLRSAP_001 for transactional data, EXIT_SAPLRSAP_002 for master data and EXIT_SAPLRSAP_004 for hierarchies.
However , consider that customer enhancement (CMOD) functionality is being converted to BADIs (RS_BBS_BADI in this case), even if not all BW CMOD enhancements have been converted to BADIs and as long as the exit will be not replaced by SAP there is no need to convert a CMOD exit to a BADI (via transaction SPAU).
In general, SAP doesn‘t recommend to use this latter ‘direct’ method to enhance extract structures in LC.
In fact, by following this procedure, changes to added fields are not extracted to BW if no additional field contained in standard extract structure was changed (since delta relevant): our ZZ* field is empty at the time of the check, in both the before and after image and, since there is no change for the system, no delta records are extracted.
And the same problem occurs in case of document deletion because document has already been deleted when the custom user exit is executed.
To do this, go to RSA6, choose your datasource, double-click on it and then on the extract structure: you will see an SE16 screen, create an append, insert your ZZ* fields, save.
Then you have to fill those fields with some ABAP custom code that can be anything from some simple calculations or table lookups to complex business logic requiring access to multiple database tables. You can do that by CMOD, creating a project and using the enhancement RSAP0001.
The function modules provided in this enhancement serve for the derivation or modification of data, that is extracted and transferred by the extraction engine of the Business Information Warehouse: EXIT_SAPLRSAP_001 for transactional data, EXIT_SAPLRSAP_002 for master data and EXIT_SAPLRSAP_004 for hierarchies.
However , consider that customer enhancement (CMOD) functionality is being converted to BADIs (RS_BBS_BADI in this case), even if not all BW CMOD enhancements have been converted to BADIs and as long as the exit will be not replaced by SAP there is no need to convert a CMOD exit to a BADI (via transaction SPAU).
In general, SAP doesn‘t recommend to use this latter ‘direct’ method to enhance extract structures in LC.
In fact, by following this procedure, changes to added fields are not extracted to BW if no additional field contained in standard extract structure was changed (since delta relevant): our ZZ* field is empty at the time of the check, in both the before and after image and, since there is no change for the system, no delta records are extracted.
And the same problem occurs in case of document deletion because document has already been deleted when the custom user exit is executed.
Getting to the root: enhance your application tables
The last resort of adding user-defined fields to extract structures is to directly enhance the document tables, in our example, VBAK, VBAP, VBEP (...).
The fields can then be filled in the general (sales) user exits for the applications and are also available in LBWE for enhancing the extract structures.
With this procedure, bear in mind that the fields added to the document tables must be saved at database level with the information they contain (with related space effort required), but, on the other side, there is no need for data retrieval in the LIS user exit.
The fields can then be filled in the general (sales) user exits for the applications and are also available in LBWE for enhancing the extract structures.
With this procedure, bear in mind that the fields added to the document tables must be saved at database level with the information they contain (with related space effort required), but, on the other side, there is no need for data retrieval in the LIS user exit.
In the end: some useful technical background info
There are four control tables involved in the customizing process in LC:
TMCEXCFS: Field status of communication structures.
Here the content is supplied by SAP: each field has a status per extract structure and communication structure: initial (inactive), A (active) or F (forbidden).
Here the content is supplied by SAP: each field has a status per extract structure and communication structure: initial (inactive), A (active) or F (forbidden).
TMCEXCFZ: Field status of customer communication structures.
In this table you can find all fields selected by the customer, per extract structure and communication structure.
In this table you can find all fields selected by the customer, per extract structure and communication structure.
TMCEXEVE: Events and extract structures.
Supplied by SAP: which event supplies which extract structure with which communication structure.
Supplied by SAP: which event supplies which extract structure with which communication structure.
TMCEXACT: datasources activation and updating status.
Also this one is supplied by SAP, but can be changed by customer.
Also this one is supplied by SAP, but can be changed by customer.
,,
