The SOA Gateway enables physical assets (tables, files, etc.) to be exposed as "atomic" WebServices, which is useful when direct access to these resources is required. However, it is often the case that a single physical table or file does not represent an object that is of use in a business sense. A typical business function will generally consist of data from more than one physical file/table/...
For this purpose, the SOA Gateway introduces the concept of Business Data Views (BDVs). This will enable the building of "compound" services around logical views of data, combining the relevant elements from the underlying physical assets. This way WebServices can be created to list, add, update, delete, get and select objects that will represent data in a business sense as against the physical view.
All businesses have enormous amounts of data which is exposed via various interfaces and applications, however, rarely, if ever, does one application, one database table or one file represent a complete piece of business related data. The reason for this is straightforward in that most IT projects tend to focus on the ‘data model’ and the efficiency of getting at data as the primary driver when designing such systems. This is still core to system design today to ensure that the physical architecture can handle the loads anticipated for a given system. However, it generally means that new projects must deal with data models that are sometimes hard to understand from a business perspective. The SOA Gateway technology now offers the ability to map the existing data and business logic assets into composite views that reflect the business view of the data. This means that newer projects can deal with business related concepts and objects when viewing or updating the core assets.
Taking a very simple example where an organization has two tables or files containing information about the owner of a vehicle and the vehicle details itself. Assume the data model here is as follows:
The Owner Table or File (The "Personnel" File)
Note:
This contains details about the owners of a vehicle. For the
purposes of this illustration, we will assume the following fields:
The owner’s unique ID which identifies the owner and will be referred to as the OID in this example. This is the primary key for this table/file.
The owner’s First Name which will be referred to as ‘Name’ in this example.
The owner’s surname which will be referred to as ‘Surname’ in this example.
The Vehicle Table or File
Note:
This will contain details about the vehicles that are owned by
owners. For the purposes of this illustration, we will assume the following
fields:
The owner’s unique ID which identifies the owner and will be referred to as the OID in this example. This is the foreign key for this table/file which links it to the "owner" file.
The vehicle’s registration number which will be referred to as ‘Regnum’ in this example. This is the primary key for this table/file.
The vehicle’s manufacturer which will be referred to as ‘Make’ in this example.
The vehicle’s model will be referred to as ‘Model’ in this example.
In the following sections, the traditional approach is discussed and the issues there are with such an approach followed by the SOA Gateway approach.
In order to access and update this data, applications must understand that there are two different tables or files and must understand the association between the files. They must also understand the requirements to keep these files consistent. In this case, when an owner is added with a OID number, the vehicles file must have a vehicle entry with that particular OID number. The access will normally be direct as per the following diagram:
There are a number of issues with this scenario:
For each request to get the data there is a requirement to make two requests to the back end database across what will normally be a busy network.
The application needs to understand that the ‘OID’ returned is used to get the associated entry in the Vehicle’s table/file. While in this case, the OID number may be of value to the client, it may not be necessary for the client to see or know this. In many cases, keys are used simply as a means to locate a unique record and have no value outside the database itself. However, these must still be understood and manipulated by the programmer.
For traditional access, software must be installed on the client system to access the back end databases. This costs time and money to keep up to date and, in many cases, has additional license costs associated with it.
If the data model is not clear to the programmers developing on the client side, the databases may end up with inconsistent data.
If the data is in multiple databases, there may be an issue with synchronizing updates to those databases unless an infrastructure supporting the two-phase-commit (2PC) protocol is available and in use.
This scenario offers a number of substantial advantages:
Only one trip to the back end system is required to retrieve or update the composite object.
The programmer does not have to know or understand internal links between tables or files and just has to deal with the fields and columns of relevance to the application.
The SOA Gateway needs to be installed on the server where the data is resident only, thus no software has to be installed or maintained on the client system apart from the client business logic itself.
The owner of the data defines the model and thus programmers always deal with the business data view and need no understanding of how the data model looks in the background.
The SOA Gateway can take care of any transactional issues with data from different databases using the transactional capabilities of the backend system. This requires two-phase-commit (2PC) infrastructure to be present on the system, the SOA Gateway will interface with it to guarantee transaction integrity.
A BusinessDataView based WebService is created in the SOA Gateway based on existing "physical" WebServices. The physical service is the service that is created to directly expose a table, file or piece of business logic. This means that existing services defined to the SOA Gateway may be reused in one or more Business Data Views. The Eclipse Control Center supplied with the SOA Gateway is used to create the business data view as illustrated in the following diagram:
As can be seen from the above, the BusinessDataView is built as a separate entity to reflect the "business entity" and is then linked to the physical data views. Some notes about the concepts:
Only the fields / columns that are required in the BusinessDataView must be selected from the physical view. In the example above, city and address in the Owner (Personnel) table/file and color (and others) in the Vehicles table/file are not used in the BusinessDataView.
Fields or columns in the physical view may be passed and parsed internally though not form part of the actually BusinessDataView. This is useful for passing internal keys to data which can sometimes be meaningless values simply to find data and thus have no value in being shown to a user.
This is all achieved using a GUI provided as part of the Eclipse Control Center with the SOA Gateway.
Once the BusinessDataView Service has been created, it can be used in precisely the same way as any other SOA Gateway WebService. If the service is called ‘OwnerVehicle’ for example, the WSDL for the SOAP requests may be returned using the following URL:
http://<host>:<port>/OwnerVehicle?WSDL
This WSDL will expose the standard database CRUD services offered today for physical services by the SOA Gateway namely ADD, DELETE, UPDATE, GET, LIST and the various SELECT methods. In the same way, to use the REST based interface, the following request will return the BusinessDataView for the Owner id ‘12345678’:
http://<host>:<port>/OwnerVehicle?F=GET&OID=12345678
All owners with an ID starting 1234 could be listed using the following URL:
http://<host>:<port>/OwnerVehicle?F=LIST&OID=1234*
The following subsections show how the BusinessDataView Service processes the various types of requests.
The SOA Gateway can make this sample BusinessDataView object available as follows using the standard REST or SOAP Based access available with the SOA Gateway:
Some notes about the above:
If the key for the vehicles file is not the same as the input key (here: OID) for the owner file it may be derived from data on the owner’s table/file.
The OID only appears once in the output but it doesn’t have to appear at all.
Any data from either table/file may be returned as part of the BusinessDataView object
Using the SOA Gateway LIST functionality, it would be possible to get a list of owners and their vehicles using this service, or all vehicles for a single owner.
When a new owner and vehicle(s) must be added or updated, the SOA Gateway can achieve this as one standard REST or SOAP request:
Some notes about the above:
Either both adds to the physical database work or neither do using a transactional semantic and thus the database remains consistent. When more than one database is involved in the transaction two-phase-commit (2PC) capabilities are required to guarantee consistent transactions.
All data is supplied in one request, this avoids the need for two round trips between the client and the server to add or update the objects (physical DataViews).
When an owner and vehicle(s) must be deleted, the SOA Gateway can achieve this as one standard REST or SOAP request:
Some notes about the above:
Either both (all) deletes from the physical database work or neither do using a transactional semantic and thus the database remains consistent.
All data is supplied in one request, this avoids the need for two round trips between the client and the server to delete the objects (physical DataViews).
While only primary keys for all affected files are required to delete the records, the same BDV used for add or update requests can be used.
The SOA Gateway offers a number of additional features in addition to enabling the creation of BusinessDataViews based on "simple" scenarios as outlined in the previous sections
When building a BusinessDataView, the data does not need to reside in the same database or even database technology. For example, the following illustrates how this would look with some data in ADABAS and some data in DB2:
Some notes about the above:
Adds, updates and deletes are also possible with this configuration but a transaction manager such as RRMS is required to ensure integrity is maintained between the databases.
Databases can be replaced and or added as appropriate without the external business logic changing.
When building a BusinessDataView, the data must not be retrieved directly from a database. The SOA Gateway can use any service defined by it as input for a business data view object. For example, the following illustrates how this would look with some data in ADABAS and a Natural subprogram providing additional data:
Some notes about the above:
Adds, updates and deletes are also possible in which case the application will be given an indication of the type of operation when it is called.
The business logic may be running in (amongst others) CICS, COBOL, PL1, C etc.
It is possible to create a BusinessDataView service based on business logic services only so no direct database access is required.
It is clear that the Business Data View concept in the SOA Gateway can provide an incredible level of flexibility to an organization in the following ways:
Avoid the need for new projects to understand legacy data models. Enable them to work with the business model which is more natural for them
Create Business Data View models from multiple sources including different databases and business logic as required.
Ensure consistency in your data by exposing services that prevent applications creating inconsistencies in the data.
Create abstract business services that may be used again and again from any technologies including Java, c#.net, vb.net, php, Ruby, Excel, Word, Internet Explorer and so on.
Secure your assets based on the business view of the data and not the physical view of the data which is where the security is really required.
Using the SOA Gateway messaging features with MQ or EntireX, enable batch updates of your business objects that can run as part of your batch processing.
To create a BusinessDataView select the target server, in the ConfigurationView select the BusinessDataViews tab
Right-click the ConfigurationView
Enter a name for the BusinessDataView to be created, click OK
Exposing BusinessDataViews as WebServices requires a BusinessDataView driver to be defined, if there is none already, proceed as follows
In the Drivers tab of the Server properties, click the Add button.
Select the BusinessDataView Driver and click Configure.
Enter bdvDriver as the "Driver Name", click Save.
Important:
For the current version of the SOA Gateway it is essential for
the correct operation of the BusinessDataView driver that it be named
bdvDriver.
Click Done to close the Driver definition Wizard.
The Adabas demo file "Employees" is keyed on personnel number and the "Vehicles" file also has a personnel number which determines which car is owned by which employee. We wish to define a BusinessDataView which gives us personnel id and surname of the employee and, if present, one instance of their vehicles car registration and date acquired.
Note:
This implements the example outlined at length in the previous
chapters.
Name the BusinessDataView "adabas_EmployeesMini_veh", click OK
This will open the BusinessDataView Editor, an initially empty "Logical View" will have been created
Right-Click anywhere within the blank area of the BusinessDataView Editor, select the Import physical WebService(s) function from the context menu
Select the adabas_EmployeesMini view, click Import and do the same for the adabas_Vehicles WebService.
Select the "Physical View" items not required for the time being, hit the <DEL> key to remove them. There is no requirement whatsoever to use all of the "base WebService" fields within the BusinessDataView's "physical resource(s)".
This will leave you with the following fields, select all fields, but NOT the personnel_id field within adabas_Vehicles. Drag them across to the "Logical View", drop them there.
This will add the selected fields to the "Logical View", and also create their connections to the source "Physical View" fields at the same time.
Select the adabas_EmployeesMini physical view.
Change the Method property to single. Do the same for the adabas_Vehicles view.
Now go to the "Logical View", double-click the personnel_id field.
Change the node name to OID.
Alternatively the node name can be changed in the properties area.
Change all field names as required, the picture should now look like this. Select Create Connections from the Palette.
Click the personnel_id field within the adabas_EmployeesMini view, this marks the starting point for the connection to be created. Now click the personnel_id field within the adabas_Vehicles view.
This "links" the "base (physical) WebServices" together and indicates the personnel_id from the adabas_EmployeesMini is to be passed on and input' into the personnel_id field within the adabas_Vehicles view. When the WebService based on the BusinessDataView is invoked, the "Employees" file is read first, the personnel_id taken from the result is then used to access the related Vehicles record.
Click the Save button to save the BusinessDataView on your SOA Gateway server
Add a new WebService, use the bdvDriver, assign a name of EmployeesVeh.
Right-click the newly defined WebService, select Publish WebService modifications.
Click the URI next to WSDL URL is ...
The WSDL for a WebService based on a BusinessDataView looks just like one for a "basic" (= "physical") WebService, exposing all available access functions (add, delete, get, list, select).
Change the WSDL directive in the URI to LIST&PID=20021*, this will show the surname for the selected Employees record(s), plus the (first) car's registration and acquisition date for every employee, where available.
We now define a BusinessDataView which provides more employee details plus a list of vehicle(s) for the employee..
Invoke the add BusinessDataView function.
Name the BusinessDataView adabas_EmployeesFull_vehlist, click OK
This will open the BusinessDataView Editor, an initially empty "Logical View" will have been created
Right-Click anywhere within the blank area of the BusinessDataView Editor, select the Import physical WebService(s) function from the context menu
Select the adabas_Employees_special view, click Import
Select the items not required for the BusinessDataView to be created, hit the <DEL> key to remove them.
This will leave you with the following fields, select the entire adabas_Employees_special view, drag it across to the "Logical View", drop it there.
You will be asked if you want to move the individual fields across, or create a group containing the individual fields.
This will add the selected fields to the "Logical View", and also create their connections to the source "Physical View" fields at the same time.
Import the adabas_Vehicles WebService.
Select the nodes which are NOT to be included in the "Logical View", as shown here, hit the <DEL> key or right-click and select Delete.
Drag the adabas_Vehicles view and drop it into the "Logical View".
When asked select to drop the view As Group.
Click the newly created group to select it, then click the group name to open ip up for editing, change the name to "car". Alternatively the node name can be changed in the properties area.
Remove the personnel_id field from the "car" group. Connect the personnel_id fields from the adabas_Employees_special and adabas_Vehicles views, channge the method properties to single and list respectively.
Finally rename the personnel_id field within the "Logival View" to PID, the entire BusinessDataView now finally looks like this.
Change the Max. Occurrences property for the address_line field to 3, for the income group field to 4.
Change the key type for the PID field to "primary"
Click the Save button to save the BusinessDataView on your SOA Gateway server
Add a new WebService, use the bdvDriver, assign a name of EmployeesVehList. Right-click the newly defined WebService, select Publish WebService modifications.
Click the URI next to WSDL URL is ...
The WSDL for a WebService based on a BusinessDataView looks just like one for a "basic" (= "physical") WebService, exposing all available access functions (add, delete, get, list, select).
We will now use soapUI (see the introduction in the tutorials section) to access the BDV-based WebService. First of all, create a new soapUI project based on the WSDL URL shown above.
Double-click the default request created for the get method, this opens up the request windows.create a new soapUI project based on the WSDL URL shown above.
Remove the entire soapenv:Header section from the request, specify aPID of 20021800. Click the submit icon to send the request to the SOA Gateway server.
The result will be a compound set of data from both the Employees (red) and Vehicles (green) files.
This section provides you with a number of general purpose hints for making your life easier in respect to editing BusinessDataViews.
When the BusinessDataView is too big, or the screen is to small to display it in its entirety at a resolution of 100%, or if you want to see some element(s) in more detail, the zoom level can be changed accordingly.
The initial display at 100%, not all details are visible.
Reduce the zoom level, from the combo box located in the toolbar area, as required to see the entire graph.
Increase the zoom level to a value above 100% to see more details.
In a situation where too many Physical Views caused the Logical View to be shifted off the visible editor area it can be braught back as follows:
Right-click the empty area, select the Reveal Logical View context function.
The Logical View will be centered in the visible editor area.
Note:
Centering the logical view will only work correctly when the zoom
level is set at 100%
Or more precisely, when the DataView associated with an imported WebService changes.
Currently there is no way of automatically reacting on changes to underlying WebService DataViews, so when new fields are added to it, and these fields are to be used in the Logical View, there is no way of updating the imported Physical View within the BusinessDataView.
The only way to deal with such a situation is to remove the Physical View from the BDV, re-import it, copy the required fields to the Logical view and recreate the links between the Physical Views.
Following versions of the SOA Gateway Control Center will provide functions to capture changes to the underlying DataView(s) without having to recreate the Physical View and the links from schratch.
