Adding custom data to Analytics - Part 1 - adding to the ETL

Starting with the Analytics 2.0\ RDB 3.0 releases a new user scripts section was added which appended a final step to the ETL process to run any additional scripts someone might have.  These user scripts could include SQL to add dimensional data to your Star that currently doesn't exist.  For example, Project Owner is a field that is related to Projects but is currently not exposed in the Extended schema or Project Dimension in Star. However if this is a field you feel you need the user scripts is the way to accomplish this. In this blog we will walk through an example of how to add Project Owner into your Project dimension. Please be aware this is just an example, not a patch or supported version.  Also the user scripts section is just an opening to allow for expansion. Make sure all scripts are approved by your DBA and are tested thoroughly for performance and data implications. If you add scripts to the user scripts folder and the ETL process begins to slow down at the final step where these scripts are executed remove the scripts and work on the sql performance.  Oracle does not support custom SQL that is not part of the product. Make sure users adding scripts to this section have the appropriate permissions. This step will connect as STARUSER unless otherwise changed. The scripts directory should only be accessible to Administrators.

Project Owner:

1- You need to add new columns to the dimension table. Add a new script to your \scripts\user_scripts folder called create_columns.sql. Here is a sample of SQL to add columns and ignore if exist:

DECLARE

  v_new_col1 number := 0;

  v_new_col2 number := 0;

BEGIN

  Select count(*) into v_new_col1 from user_tab_cols where column_name = 'PROJECT_OWNER' and table_name = 'W_PROJECT_D';

  if (v_new_col1 = 0) then

      execute immediate 'alter table W_PROJECT_D add PROJECT_OWNER varchar2(255)';

  end if;

  Select count(*) into v_new_col2 from user_tab_cols where column_name = 'PROJECT_RSRC_OWNER_OBJECT_ID' and table_name = 'W_PROJECT_D';

  if (v_new_col2 = 0) then

      execute immediate 'alter table W_PROJECT_D add PROJECT_RSRC_OWNER_OBJECT_ID varchar2(255)';

  end if;

  commit;

END;

/

EXIT

In this example, we need to do some transformation of the data. In the Extended schema in the Project view there is a field for "ownerresourceobjectid" this is the object id of the resource that owns the project. For some this may be enough, but you probably want to have the actual username for the resource that owns this project. Because of this we are going to create 2 columns in the dimension but only expose one to Analytics. The first column will store the project resource owner id, then we will find the username based on this object id and update this other new column for project owner with the username.

2- Now we need to populate these columns. Add a new script to your \scripts\user_scripts folder called project_owner.sql. Here is a sample of SQL to populate these fields:

set serveroutput on size 100000

-- add project owner values

declare

  v_src_id number;

  v_link_name varchar2(25);

  vsql varchar2(200);

BEGIN

  v_src_id := get_source_db_id;

  v_link_name := get_link_name;

vsql := 'update w_project_d set PROJECT_RSRC_OWNER_OBJECT_ID = (select ownerresourceobjectid from project@' || v_link_name || ' where objectid = w_project_d.project_object_id) WHERE datasource_id =' || v_src_id; 

execute immediate vsql;

UPDATE w_project_d set PROJECT_OWNER = (select user_name from w_resource_d where resource_object_id = w_project_d.project_rsrc_owner_object_id) WHERE datasource_id = v_src_id;

commit;

end;

/

Exit

The content of this SQL is just an example. What we are doing in the first update is getting the ownerresourceobjectid from the Project view in the Extended schema through the database link.  Next we are going to match based on the resource object id and find the username from a table already  existing in STAR. Then we will commit and exit out of the process.

3- We need to define the order the scripts are run in the user_scripts folder.  In P6 Reporting Database you can define order that scripts get executed.  This is helpful in a case like this or if you see deadlock issues.  Go to \res\priority\ and edit the user_scripts.txt (if it does not exist, add it).  Set the following priority:

# Order of user scripts

create_columns.sql

project_owner.sql

This is a basic sample of how to add data to your STAR data warehouse that may not be part of the product. Keep copies of these scripts so when upgrading you do not lose them. The user scripts concept has continued into the more recent versions of the P6 Reporting Database\P6 Analytics but during upgrade and new installation these scripts will need to be re-added to your new installation.     Upcoming will be another blog about part 2 of this process, how to get this new data to show in P6 Analytics.