Browse Author

beyerch

Technology expert, voider of warranties, mad scientist.

HFM – Removing Years

One of the most important steps during the creation of your application creation is determining how many years should exist in the application.  Specifying too many years could result in having wasted space and poor application performance while too few years could result in running out of room in your application when you hit the upper limit.

HFM Application Profile Editor (Pre-11.1.2.4 shown, though same applies to 11.1.2.4)

When HFM first shipped, you had no means of updating the Start Year or the End Year of your application; therefore, a mistake during the application creation process could lead to major headaches in the future.  As customer’s HFM applications matured and those upper year limits were getting close, Hyperion/Oracle rolled out the capability to add more years to the applications.  (with 11.1.2.4, there is a built in function to do this)

When it comes to removing years from the application, you can clear data from years; however, there is not a way to actually remove the years in the application.  Unfortunately, Oracle has never released guidance or functionality for altering the Start Year and will tell you that it isn’t possible to change this.  I’m here to tell you that, you can absolutely change this!  Before we get into how to make those updates, some quick words on why you’d want to do this and some considerations around it.

Reasons to remove years from your application:

  • Database has grown significantly due to continued use of the same application over many years
  • Metadata is cluttered due to the need to ensure historical data remains consistent
  • Prior years are no longer used for reporting  and are not necessary
  • Minimize the years displayed in the U/I to only valid years
  • Reduce the number of database objects (data tables may exist for each year/period/scenario combination)

Considerations when removing years from your application:

  • Beginning Balances for Balance Sheet accounts – (e.g. If our Start Year changes to 2015, we will still need prior year ending balances)
  • HFM Rules/Calc Mgr – (e.g. ensure that rules are not impacted by beginning year change

Process to change the Start Year:

  • “Easy Steps”
    • Make a backup.  (LCM / Database Backup / etc.)
    • Create an archive application – Consider creating an archive HFM application for the historical data.  Storing the information in a separate application gives you the ability to still view the data if needed without having to make metadata trade-offs in your day to day/primary application.  Use HFM Application Copy Utility or LCM/Application Snapshot to accomplish this.  (If you already have an archive app, you can export/import data, though Journals may be slightly annoying in this instance)
    • Update Beginning Year variables / values in the application rules / data for the new Start Year
    • Stop the HFM services
  • “Not so Easy Steps”  (explained later in more detail, don’t panic yet)
    • Connect to your HFM database via your database tool of choice  (e.g. SQL Server Management Studio, SQLPlus, etc.)
    • Query the <APP_NAME>_BINARYFILES table by ENTRYIDX for Labels of type SharedAppData
    • Update bytes 74 & 75 of the first row to reflect your new Start Year
  • Start HFM Services
  • Verify application data / functionality
  • “Post Validation Cleanup”
    • Stop HFM Services
    • Remove Year / Period specific database tables (e.g. <APP_NAME>_DCE_<PeriodNum>_<Year> )

HFM Binary Files Table

For each HFM application, you will find a Binary Files table in the format <APP_NAME>_BINARYFILES.  This table is used for multiple purposes, including storing application configuration information, application metadata, HFM rules, memberlists, etc.  The LABEL column defines the type of file the data is related to.  The BINARYFILE column contains the raw information from the corresponding file.  (e.g. Application Profile .PER file, Rules .RUL file, Metadata, etc.)  Since this column has a character limit that is smaller than most of these files, you will find multiple rows for each file type.  The ENTRYIDX column is used to order the rows in the proper order.  (Ascending order)

As the Start and End years are defined in the profile, we only want to view rows containing a LABEL of SharedAppData:

SELECT
 ENTRYIDX, LABEL, BINARYFILE
FROM
 <APP_NAME>_BINARYFILES
WHERE
 LABEL = 'SharedAppData'
ORDER BY
 ENTRYIDX ASC

 

SharedAppData 

SharedAppData ENTRYIDX 1 as viewed in a Hex Editor

As mentioned above, the SharedAppData rows correspond to the application profile (.PER) that defines the HFM application.  Analysis of this data indicates that both the Start and End years are stored in the first row of this information.  The Start Year is stored in bytes 74 and 75 while, the End Year is stored in bytes 78 and 79.  The data is stored in Least Significant Byte Order.  (e.g. Byte 75 is first half of year, 74 is the second)

In the Shared App Data above, the Start Year is 2006 while the End Year is 2015.

Start Year (Hexadecimal) = D6 07 = 07 D6 (flipped) = 2006 (decimal)
End Year (Hexadecimal) = DF 07 = 07 DF (flipped) = 2015 (decimal)

To change the Start Year, convert the decimal year to hexadecimal and then reverse the byte order.  For instance, if your new start year is 1999, you would end up with:  1999 (dec) = 07CF = CE 07

HINT:  You can use Windows built-in Calculator program in Programmer mode to make these conversions.

Update SharedAppData

After you’ve calculated your new Start Year value, create an Update query to alter the stored binary data

UPDATE query to alter Start Year

 

Verify Results

After you have updated your settings and restarted HFM, you can verify that you now have the appropriate years

Original Years

 

Updated Years

NOTE – While the screen shots are from an HFM 11.1.2.1 application, this has been tested through 11.1.2.4, though I won’t guarantee this works with every patch version, etc.  Always test in a non-production environment.

Hyperion Profitability (HPCM) 11.1.2.4 – “Transaction rolled back because transaction was set to RollbackOnly”

While working with a client running HPCM 11.1.2.4.110, we were encountering intermittent errors while executing rules:

Problem Overview

“javax.persistence.RollbackException: Transaction rolled back because transaction was set to RollbackOnly”

1_UI_Error

As the exact same rule could be re-run without any errors, it appeared to be application/environment related.  (also appeared to be database related given the content of the error)

Reviewing the profitability log provides a much clearer view of the issue

NOTE: Log would typically be found in a location similar to:  \Oracle\Middleware\user_projects\domains\EPMSystem\servers\Profitability\logs\hpcm.log

3_HPCM_Error_Log

From the log file snippet above, note the highlighted section:

“[SQL Server] Arithmetic overflow error converting expression to data type smallinit.”


What is the Problem?

While this does not match the error message we see in the user interface, this is definitely related as:

  1. The error message was logged at the same time as the error in HPCM
  2. The user name logged corresponded to the user receiving the error
  3. SQL Server rolling back a transaction after a failed insert makes a lot of sense.

(very) Loosely speaking, database transactions exist to protect the integrity of a database.  If a program, or user, were to execute a series of statements against a database and one or more fail, what should happen?  Should we leave the database in an inconsistent state or should we put the database back and alert the user?  While application developers could build this logic into their program code, it is a lot more convenient to give the database a series of steps and let it handle that for us!

In this case, the INSERT statement is part of a transaction.  Since the INSERT failed, SQL Server has rolled back the entire transaction and reported that to HPCM.


Why are we encountering this problem?

While that explain what happened, why did this happen?  The error in the log file has four key clues :

  1. We are attempting to add data to a database table  (INSERT INTO)
  2. The table is: HPM_STAT_DETAIL
  3. ARITHMETIC OVERFLOW occurred when trying to store a value in a column
  4. The target column has a Data Type of smallint

In SQL Server, a smallint datatype can have a maximum value of 32,767.  Another look at the error message reveals one numeric, 43,014, which exceeds 32,767.  This value is being stored in a column called JAVA_THREAD.  As JAVA_THREAD is storing the process id, which is semi-randomly generated, if the number returned is < 32,768, the program works as expected.  If the ID is > 32,767, then things don’t go as well…..

Reviewing the table structure for this table confirms the suspicion.

2_Database_Column_Definitions


How to fix this

The easiest fix for this issue is to change the datatype for this column from smallint to int.  As the largest int value is well over 2 Billion, this issue should not occur again.

LEGALESE – While I have reviewed this change with Oracle and am very confident this will not cause any issues, proceed at your own risk.   🙂

4_Updated_Table

NOTE(s):

  • As of 6/26, Oracle has confirmed this as a BUG.  No ETA on an update yet, though.
  • This may be SQL Server specific, have not evaluated against Oracle schema to confirm data type used.  [Oracle equivalent of smallint would be number(1)]

2004 Pontiac Grand Prix GTP Turbocharger Installation – 401 HP Out Of 231 Cubes

Since you can never have enough horsepower, I worked with my good friend, Kevin Margitta of Cartuning to replace the Eaton Gen IV M90 Supercharger on my 2004 Grand Prix Comp G with an intercooled turbocharger.  While Kevin had the hardware worked out, I contributed to the project by creating the software necessarily to reprogram the engine’s control module.  With the intercooled turbo & software updates in place, this swap easily put down 401 HP to the wheels before I decided I had enough fun at the dyno shop for the day.

A couple of months after the swap was complete, I ran into Don Keefe and asked him if he would like a ride in the car.  While he didn’t seem very interested at first, he became very interested after the boost kicked in!  Shortly thereafter, the following awesome article was published in High Performance Pontiac:

http://www.hotrod.com/how-to/engine/hppp-0603-2004-pontiac-grand-prix/

0603_hppp_01z+2004_pontiac_grand_prix+launch
Getting ready to make an 1/8th mile pass

 

0603_hppp_02z+2004_pontiac_grand_prix+blower_casing
Engine Bay View Post Turbo Install

 

0603_hppp_03z+2004_pontiac_grand_prix+turbo_location
Turbo Installation Closeup

 

0603_hppp_05z+2004_pontiac_grand_prix+hollow_supercharger
Supercharger Rotor Block-off Plate

 

0603_hppp_06z+2004_pontiac_grand_prix+stock_manifold
Another view of Turbo / Inlet piping

 

0603_hppp_07z+2004_pontiac_grand_prix+intercooler
Air-to-Air Intercooler

 

0603_hppp_08z+2004_pontiac_grand_prix+wastegate
Turbonetics Wastegate

 

0603_hppp_12z+2004_pontiac_grand_prix+turbo_kit
Cartuning Turbo Kit Display Model

 

0603_hppp_13z+2004_pontiac_grand_prix+sleeper
Stock Appearing Car

 

 

 

 

Electronics Project Room …..

 

I’ve been slowly repurposing a spare room in the basement to serve as my electronics project room.  Not quite done, but thought I’d post some photos showing current status.  Just need to get the electrical finalized and will be good to go!

 

 

Primary Work Area
Bench #1

 

Decorative / Functional Gauges
Decorative / Functional Gauges
Oscilloscope & Stuff
Oscilloscope & Stuff
Trusty Hakko 936
Hakko 936 Soldering Station & EMP 20 Programmer

Kscope2015 – Smart View

This past year at Kscope15, I presented a session all about the technical side of Smart View.  While I intended on spending equal time between infrastructure and API/programming  topics, I ended up focusing a bit more on the API/programming side.  There are so many ways to improve your Smart View documents by understanding some basic Visual Basic for Applications (VBA) code and leveraging the Smart View API, I simply couldn’t resist!

For those more interested in the infrastructure side of Smart View deployments, do not fear!  While the session itself didn’t spend as much time on it, the Power Point includes a fair amount of slides which provide information on how to automate Smart View deployment, automatically set user defaults, and deploy preconfigured Private and Shared Connections.

The sessions, and slide deck below, provide oodles of information on the following topics:

  • Improving Robustness of Smart View Documents
    • Excel Add-In Failure Detection  (e.g. Disabled Add-In / Missing Smart View)
    • Proactive Connection Monitoring
  • Deployment Simplification / Initial Configuration
    • Automated Installation Guidance
    • Automated Default Preferences Push
    • Automated Shared / Private Connection Push
  • Essbase Add-In / Smart View Conversions
  • VBA Important Tips / Tricks
  • Smart View API Important Tips / Tricks

As with all of my presentations, you will find a plethora of working examples such as:

  • Excel Performance Improvements ( Screen Updating / Enable Events / Calculation Mode )
  • Invalid Cell Data Identification ( Catch Non-Numeric data before it wrecks your formulas! )
  • Add-In Presence & Status Detection
  • Broken Link Detection & Correction
  • Planning Cell Note Editor
  • Working with Excel & VBA (Workbooks / Worksheets / Ranges / Events )
  • Working with Smart View API ( Refreshing Data / Creating, Establishing, Disconnecting, Deleting Connections )

Download the presentation here!