The last large session of the day was the Technical Roundtable. Chris Heller and Larry Grey of Grey Sparling were asked to lead the discussion because of their extensive understanding of PeopleSoft and the underlying architecture. I filled in for Larry and mainly took notes since its been a while since I worked on the PeopleSoft products.
The issues brought to the roundtable break down in to the following categories: Security, version upgrade via service pack/bundle/maintenance packs, performance issues, problems with Crystal Reports formatting during upgrades, release testing and quality.
No one attending was using LDAP security but there was one person who was planning on testing it to see if using it would help solve some security issues. The first tip was to setup security at each level that requires security with the same user and password combination. This is fine for initial setup but maintenance of passwords and disabling accounts can get tricky.
Don’t delete PeopleSoft user profiles. Doing so ‘changes history’ by removing the who on the data in the system and there was confusion if PeopleSoft even lets you do that.
Since the security in PeopleSoft is exposed as part of the API, you can write scripts against the security in PeopleSoft to enable you to do bulk changes to security attributes. One example of a use for this is in the termination process of HR. Since large companies can have many employee terminations on the same day, it makes sense to create a script that can take the list of employees to be terminated and update the security to prevent the user access to the system. It takes the onus off of the HR employee who is probably manually doing that now. However, don’t forget to audit the changes!
Several issues of performance came up in the session.
WebLogic 8.1 bug affecting performance
Apparently, there is a bug that affects performance in the WebLogic application server layer that was introduced in WebLogic version 8.1 which is corrected in a later service pack (sp8 or sp9). This bug can generate enormous error logs on the application server. So large that they can consume the entire disk on the application server (thereby causing even more errors).
Another performance problem can be caused by users. If users become impatient with the application and begin to hit buttons multiple times (ex: the browser back button, run report, etc.), each button will generate a request for service to the application server which will try its best to service all of the requests, not just the last one.
The answer to that question is a resounding NO. This is a general issue for hosted web applications. The resource requirements for a database server are very large as are those for an application server. If both are on the same physical machine, they will starve each other of needed resources.
The default installation will contain demo certificates for the application server which need to be replaced after installation. Otherwise, the application server will continually add to the error log with messages about the demo certificate. Note this may be related to the WebLogic bug mentioned above.
Since no system no matter how well tuned and adjusted will have perfect performance, it helps to have a monitoring regiment in place to ensure that it is working well on a day-to-day basis. Some recommended applications were:
One other tip was not to use PS trace in the App Engine. Also PS-PerfMonitor is very time intensive to use.
SQL Server can sometimes get deadlocks (esp. when doing batch processing at peak times). In those instances, the Enterprise Manager is the solution. It identifies the deadlocks and can free them in real-time.
There were a set of problems and clarifications around Crystal Reports.
There were a couple of instances of problems, neither of which found a good way to fix the problem other than to manually re-format the reports. The first occurred after an upgrade and affected the behavior of the micr fonts. In essence the micr font used on a set of checks enlarged which shifted the information on the check. At issue may have been the fact that the font size was customized to a fractional point in order for the check to print correctly. The second occurred after a PeopleSoft upgrade (7.5 -> 8.1). The formatting in the reports simply didn’t stay put. This may have been the result of a change in the Crystal Reports version shipped with PeopleTools. In both cases the solution was to reformat the affected reports manually.
Another problem centered around the export to Word function in an older version of Crystal. When the report was exported, the resulting Word document had incorrect line spacing making the result unusable. No solution for this one.
PeopleTools depends on the name of database objects to do its magic. So if you install the 8.4x PeopleTools instances on a different database than the currently running 8.1x version, you can use database linking to link the 8.4x metadata to the 8.1x data and thereby create an environment where the 8.4x PeopleTools reporting tools can access the current 8.1x data.
The Crystal Enterprise version shipped with PeopleTools is a limited 5 user license. These 5 users are concurrent runtime licenses not just development licenses (as mentioned by Chris in the discussion). In order to use a more expansive license for the Crystal Enterprise product you will need to contact your Sales Representative from Oracle or Business Objects.
Upgrading is always an interesting topic and this session was no different. Everyone seemed to have something to ask about upgrading PeopleSoft applications and Tools. Aside from the issues involved with upgrading Crystal Reports (as mentioned above), there were questions ranging from upgrading major PeopleSoft releases (7.5->8.x) and when should you upgrade via bundles or maintenance packages.
Since both state and federal payroll taxes can change during the year, if you are going update the payroll tax tables anyway, it makes sense to bring in and of the upgrade bundles that you are missing. Remember that auditing the changes is always a good idea.
This one really doesn’t have an answer. The best you can do is decide if the upgrade has enough merit to out-weigh the cost of testing the upgrade. This really depends on your immediate need. If you take individual bundles, you may start losing support from Oracle by virtue of the fact that you are venturing into un-supportable configuations. There will come a time when the ChangeAssistant starts giving you so many possible updates that Oracle loses confidence in the stability of the system (at least from an Oracle support standpoint!). When this becomes an issue you will (eventually) either need to upgrade to all of the supported bundles, or live with the fact that Oracle can no longer support your system. (Don’t despair, read a little further to the next section or click here)
Yuck. The big problem with bundles (its also a feature) is that you are able to select which (potentially system damaging) updates to apply. If you do this often enough and you call Oracle support you will get the dreaded question, “Are you up-todate?”. The answer is usually: “No, I’ve only been applying the bundles that apply to my situation and am afraid to do a complete maintenance pack.” In that case support will probably cut you loose saying that you are “in an unsupportable configuration…” or some other nonsense. This leaves you in the position of saying, “… well, I’m not sure that we are up-to-date, but I think so.” or trying to fend for yourself and upgrading the system to the current set of maintenance packs.
ChangeAssistant is the link between you and Oracle support. If ChangeAssistant says you’re up-to-date, then you are. The basic idea of ChangeAssistant is to create an environment whereby you can check to find updates and download them in bulk. The benefit is that it removes the tedium of doing the checks manually, but at the same time it can be detrimental to the behavior of the system. The best advice is to use ChangeAssistant to update your PeopleSoft demo database. Since the demo database is not part of the production system, you can use it as a staging point for updates and still tell Oracle that you are “up-to-date”.
One of the strongest sentiments came as the question, “Why can’t you [Oracle/PeopleSoft] test the simplest functionality that I need before you release?” That is a powerful question in all of software development and one that many engineers lose sleep over. The answer is basically, “We can’t test everything.” A better answer is that software customers should follow the edict “caveat emptor”, let the buyer (you) be aware of what you are accepting from the vendor (Oracle/PeopleSoft). All software companies with which I have been associated (PeopleSoft included!), try to make the customer’s experience with their software positive. It is true that sometimes release dates can intrude on the quality of the software, however, the whole point is to make the customer more productive. The best tip shared for testing was to ensure that you (as the customer) have quality assurance tests that you feel you can trust and be able to run them on the PeopleSoft demo database or a static database that is functionally equivalent to your own either manually (not suggested) or automated (definitly a good idea). Then by downloading the changes via ChangeAssistant to the PeopleSoft demo database you can control the acceptance testing by running your acceptance tests.
Since Fusion is a product in development, we can only use the publicly available information as source. It looks like the WebLogic and WebSphere servers will be unsupported in Fusion. As for SQR and COBOL, remember that the SQR and COBOL in PeopleSoft are stable in the sense that both of the code bases for those languages have been included in the product for several years. They are not dependent on the language modifications imposed by the current providers.