BW on HANA: Working of In-Memory DSO

How does today’s Standard DSO work?


Data Store Objects are used for staging purposes and also for operational reporting in SAP BW. The Standard DSOs consists of 3 Tables Active table, Activation Queue and Change log table.

Activation of DSO means calculating the delta and updating the current data snapshot into Active table and writing the history of changes made into change log table.
We can see a detailed flow of the same in the following diagram:

Let us take an example scenario:
We have an ATM1 in which we have a initial balance of 400 and an delta load of balance 100.


How activation of Request takes place?
The request activation takes place on BW side and saves the changes to the database as depicted in the figure below:
 :


Where is the today’s DSO performance critical now?
As discussed above, we have 3 tables which DSO uses for calculating Delta. Application Server performs delta calculation and executes every SQL statement in the database. Thus for every calculation Application server needs to communicate with the DBMS, thereby creating heavy load on RDBMS server. These round trips makes the activation time very high, which takes more than 80% of runtime.
As we know about HANA computation power, SAP has decided to move the calculation part to SAP HANA DATABASE to optimize the activation time.

What is IMCE (HANA) and what do you need to know about column store?
Main Storage:It contains main parts of the data which is not only highly compressed but also read optimized.
History Storage:It helps in versioning of data (History).
Delta Storage:
It exists only in main memory. Change operations are stored here and are used for delta management.
Block diagram of Delta Management in Column Store:


What is Delta Merge operation?

• It moves data from Delta Storage to Main Storage because Main Storage is read optimized.
• It uses MVCC to maintain the consistency in the database for read & write operations. MVCC is discussed in detail in the later sections.

Write Operation:

• It is used only on delta Storage as it is write-optimized.
• The updating of database is performed by inserting a new entry into delta storage.

Read Operations:

• It reads from Main & Delta storages and merges the results.
• IMCE uses MVCC (Multi Version Concurrency Control) to ensure consistent read operations.

How this architecture with BW does improve the activation times?
The request activation along with updating of database takes place on SAP HANA DB side as depicted in the figure below:

This means there will be no more roundtrips to database for saving the updated information. As the activation is taking place in the database itself. Hence reduces the activation time.
We can make DSO as an In-Memory based DSO only if it is of Standard type.
Only DSO of type “Standard” can be converted to Standard DSO. You can find the same in the screenshots below:

Standard DSO:


You can see the same option is not available for other types of DSO’s.
Direct Update:


Write-Optimized DSO:


What is MVCC?
Multi Version concurrency control (MVCC) for ensuring consistent concurrent access to the real time databases.

The purpose of MVCC is:

• To ensures database consistency.
• To resolve read-write and write-write conflicts.
• To prioritize transactions and avoid deadlocks.

How does MVCC work?
We have a Table in database which looks like this:

Object	Value
A	10

Let us consider Read operation as R (A) and Write operation as W (A).The timestamps registered, are used by MVCC to lock and ensure consistency. Ti, Where i = 1,2,3,4 and 5 (Timestamps 1<2<3<4<5)
The order of transactions takes place as the below Diagram:


In the above manner, MVCC ensures consistent read operations.

How does In Memory DSO work?
In IMCE, the request activation takes place at IMCE side as shown below. And BW server will have a view on these tables at SAP HANA DB.
InMemory DSO consists of same tables i.e. Activation Queue, Change log and Active Data table. Along with control tables /BIC/A<ODSNAME>70 and /BIC/A<ODSNAME>80.
Change log and Active Data table consists of additional field IMO_INT_KEY.

Request Activation in InMemory DSO:
Below figures will depict the process of request activation in InMemory DSO.






Now let us take the same example of a sample transaction which was discussed earlier using Standard DSO.
Example: We have an ATM1 in which we have an initial balance of 400 and a delta load of balance 100.
Step 1:
:

Step 2:


Hope you understood the working of InMemory DSO.
Now let us discuss migration concepts of DSO to InMemory DSO. Different options available for migration:

1. Simple Migration
2. Full Migration

1. Simple Migration:
Only Active data gets converted and change log history gets deleted. Conversion is faster and requests cannot be rolled back after migration.
2. Full Migration:
Active data gets converted along with change log history. Conversion is slower but there is no restriction on rollback of requests.
This is about HANA Architecture with BW and how it improves performance with SAP BW and working of InMemory DSO.
We will discuss about working of In-Memory Cube, in my next upcoming blog.

Now let us discuss migration concepts of DSO to InMemory DSO. Different options available for migration:

1. Simple Migration
2. Full Migration

1. Simple Migration:
Only Active data gets converted and change log history gets deleted. Conversion is faster and requests cannot be rolled back after migration.
2. Full Migration:
Active data gets converted along with change log history. Conversion is slower but there is no restriction on rollback of requests.
This is about HANA Architecture with BW and how it improves performance with SAP BW and working of InMemory DSO.
We will discuss about working of In-Memory Cube, in my next upcoming blog.