Compiere from the Source

Migration Challenges

Version Migration allows to move your installation to the current distribution version to take advantage of new features, improved performance and stability (refer to other postings for Component/Application migration and Data migration).

The version migration is stable in production since 2003 but has the disadvantage that it is the barer of bad news.  The migration does not create issues - it discovers and identifies issues in your current system.  In this blog, I will discuss typical problems and your options of solving them.

One important principal: Errors will not go away - they will compound.  So fix them ASAP. Procastination will ensure that you'll spend significantly more time fixing them later and might even destabilize the system.

When evaluating the situation, view all of the errors displayed.  You very often find that ther are just a few causes creating potentially a long list of errors.  Example: if some record fails to be inserted due to a naining constraint, you may also get many other insert errors failing due to parent contstraints.

Column Name Conflicts

Cause: you added a column in a Dictionary table and get something like the following error:

ORA-00001: unique constraint (COMPIEREPRO.AD_COLUMN_NAME) violated
INSERT INTO AD_Column (AD_CLIENT_ID....)
 VALUES (0,15851,550,0,19,333,236,'C_Campaign_ID','R',TO_DATE('2006-07-07 17:19:08','YYYY-MM-DD HH24:MI:SS'),0,'Marketing Campaign','D',10,..)

In this case an entity with the same name was created by the user.  So, identify that entity and in general either rename it - or delete it in the application.  Here, a custom column already exists and you probably do not want to loose the existing data.  So here, the solution would be to delete the column in the dictionary - this does not delete the column and information in the database.  The migration process will then add the column again, but not change the database as the column already exists. 

Note that Compiere column names are case sensitive whereas the column name in the database may not be case sensitive (default in Oracle, other databases have case sensitive column names by default).  In this case, you get a slightly different error - adding the duplicate column in Compiere succeeds, but creating it in the database fails (ORA-01430: column being added already exists in table).  The sulution is the same.

Callout for Swing and WebUI

In Compiere, you use a Java Callout for "data entry consequences" - example - you select a business partner, which then populates business partner location and user as well as other specific default values like payment terms.  Without a Callout, the system would just set the dependent values (location, user) to null.  Consequently, Callouts are essential when you want to provide easy-to-use, interactive windows.

Callouts are available for Compiere for a long time and one of the reason for Compiere's success.  The ultimate objective is to have a single controller for Swing and the WebUI with one Callout API.  At this momemnt, you need to use platform specific code to create a callout.

As a example, I will use the situation to automatocally select a revenue recognition method after you selected a product - i.e. after you enter the product, the system checks if revenue recognition applies and selects the appropriate mathod, which you the could overwrite.  Revenue Recognition e.g. distributes the revenue you cannot recognize immediate, but over time.

Data Dictionary

After logging in as System Administrator, open window "Table and Column" - navigate to the table and column you want to enable - here you find the Callout flag and the Callout code.

Callout for Swing UI

The steps to create a Callout for the Swing UI are:

• Create a class which extends CalloutEngine
• Create a method with the following signature:
  public String justAnExample (Ctx ctx, int WindowNo, GridTab mTab, GridField mField, Object value, Object oldValue)
• Enter the fully qualified class.method name in the Data Dictionary Callout code for the column 

The business logic you provide here should be repeated on the model level to have the same experience/behavior if you were to enter/modify a record manually via UI or via API.

There are many examples for callouts in Swing - check classes like CalloutInvoice for a bit more complex logic or the class CalloutUser for basic information.

Callout for WebUI

The objective was to have all functionality in the model class, so you would just add a special setter method - example from MConversionRate:

@UICallout public void setMultiplyRate (String multiplyRateOld, String multiplyRateNew, int windowNo) throws Exception
{
  setMultiplyRate(convertToBigDecimal(multiplyRateNew));
}

You would then just convert the String to the correct data type using a static method and call a standard method.  This way, there is no need to replicate functionality on the model level as this is executed on the model level.

To enable the callout, you just select the Callout flag for the Column in the dictionary.

This approach is easy, if you are the owner of the object. If you want to extend an existing class, you have two options:

• If you create a component, you just enter your model package in your entity type and create in there your extension in your package - example:
  public class MOrder extends org.compiere.model.MOrder
• Use an external class (details below). You may want to select this option, if you don't want to create your own extension, e.g. because another component is alreay doing that - or becuase the change is simple

To create an external [external to the model class] WebUI callout, you just create a method in a class with a parameter-less constructor and add the fully qualified class.method name in the Data Dictionary Callout code for the column - example:

public void justAnExample (PO po, UIField field, String oldValue, String newValue)

Note that you can use the same class.method name for SwingUI and WebUI - here compiere.model.CalloutUser.justAnExample - as the parameters are different, it is easy to differentiate and allows to have the code in the same class. Note that you should use the ModelValidator to replicate the logic for the model level to achieve the same behavior via API.

Web Store Management

Compiere can easily link to third party web stores, and also provides self-service and web store functionality.  In addition to the web store functionality of viewing and buying products, Compiere provided comprehensive account management functionality.  In addition to address management, the customer or vendor can view their orders, invoices, shipments / receipts and payments, pay any open order/invoice or create pre-payments as well as enter/manage (customer service) request.

To set up the web store, you need to configure the mail server (Compiere also supports TLS) and the credit card processor (support PayFlowPro out of the box and can be easily extended to support Authorize.net or any other payment processor).  We also recommend you set-up Apache as a front-end server for ease of security management security and access.

For ease of customization, the Compiere web store uses Java Server Page (jsp) technology designed to work easily with any browser.  The first level of customizations are cascading style sheets (css).  Prepackaged are three style-sheets: gardenworld.css, compiere.css, standard.css, but you could also reference any external style sheet.  This is done in the web store definition window in the application. There you also determine what functionality to expose (like Request for Quotation, Requests, ..) and what logo to display.  Here are also some additional options to add text or links in different predefined regions of the web store layout.

The next level of customization would be to modify the the jsp files.  Some reasons for this type of customization would be to change the content and sequence of the individual fields.  Example: show the available quantity of a product or to show a flag "in stock".  To do this, you would use the compiereWebStoreBase.war file in the /lib directory in the Compiere home.

The deployment of the web store is done during setup.  When executing RUN_setup, the compiereWebStoreBase.war file is then combined with the appropriate code file and the compiereWebStore.xml file to the compiereWebStore.ear file and automatically deployed to the application server.

Compiere support multiple tenants. So how can you create multiple web stores for a single Compiere instance?

Compiere Deployment on JBoss, Tomcat, OAS and WAS

Compiere is designed to support generic J2EE Application server, but as usual, there are some detail considerations.  Out of the box, Compiere supports JBoss and manages it's configuration and deployment.  In addition to JBoss, you can deploy the full application server to IBM WebSphere (WAS), Oracle Application Server (OAS) and others in addition to deploying the web containers in Tomcat.

In the setup dialog, you have the choice of "jboss", "tomcatOnly" and "ibmWAS" (the "oracleAS" option is disabled at the moment).  In this blog, I will describe how best to implement the different options.

Let's start with stand alone Tomcat. You can deploy war (WebArchive) files to Tomcat, examples are the web store, the web user interface.  You may do that for performance reasons and you always need another application server for the full functionality.  When selecting Tomcat in the Compiere Setup, you need to provide a path to your Tomcat deployment directory.  When starting Tomcat, you need to make sure that you add the Compiere Home variable to the Tomcat start command
(example: -DCOMPIERE_HOME=C:\Compiere2)
and you must configure the port to listen on.  I suggest that you rename the war file as the context provided is the exact name of the war file.

Aspect Oriented Development in Compiere

Aspect Oriented Programming (AOP) is an approch to manage functionality (aspects) of crosscutting concerns. The dedicated development environments allow the user of a developed functionality to add join points in method calls, etc. to add new behavior or even overwrite current behavior.  Aspects are a great way to isolate customisations and extension functionality.

Compiere is developed in a "classic" non-aspect oriented Java environment, but allows the user to "plug-in" their extension quickly without the need to modify or extend the underlying class.  This eases development and certainly migraion of that functionality to new versions or distributing that functionality to multiple sites.

Compiere provided the following join points for quite some time:

• Before Save (new or update) or Delete of any Persistent Object (PO)
• Before the workflow step Prepare or After the workflow step Complete
• After the user and role is identified in the Login process
• Before a Info window is displayed to modify or add information columns

Automatic Value Assignments

Compiere has the ability to dynamically create default field values when a new record is created (SQL, constant, preferences).  In addition to this, Automatic Assignments allow to set values after the user pressed save, just before a record is saved.  This could be used to create a default value without user intervention or if the user fails to define a value.  Another use is to assign a certain value based on other fields. 

Automatic Assignments are basically declerative before-insert/update/delete trigger and the main use is to complete records based on user entered values, set status, routing.  It is an easy alternative to extending the business class and modifying the beforeSave method.

Examples include: In a Lead assign the sales rep based on the country/region of the lead;

The Automatic Assignment allows you to add business logic without programming nor impact on migration to new versions and with that an ideal Customization tool. It acts like a Trigger and it's execution is logged.

You define the auto assignment for a certain table and select the situation the assignment should take place:  e.g., when a record is Created and/or Updated. You can add the restriction that the record cannot modified if processed (for documents are assumed to be read/only after processing).

Migration Tips & Tricks

Compiere provides three types of Migration

• Version Migration - Moving your system from one version/release to another
• Component Migration - Installing and upgrading your Component/Customization
• Data Migration - Moving/updating selections of data from one system to another

In contrast to other ERP/CRM systems, Compiere was designed to be extended/customized and with the principle that there is only one maintained version - the latest. In the beginning people were a bit uncomfortable with the idea that there is no support for historical versions. When they experienced the benefits of the Version Migration (available and stable since 2001), this concern was no longer an issue.  To be able to migrate to the current version new concepts were introduced to enable this.  The main element is the clear distinction who owns what via the Entity Type - and that in the migration does not touch customizations nor customer data.

The Version Migration is basically a "diff" program which compares the current state of the target system with the to be achieved state of the source system.  The migration is now in it's third generation and also maintains database index, constraints and views.  The runtime is more dependent on the number of differences between the versions rather than the size of the database.

The main advantage of the migration is that it uncovers potential issues (it does NOT create them).  The most useful output is the error log.  In most cases, you can start/continue production, even if you have a significant number of issues.  Let's look into causes and options:

Compiere Scaleability & Performance

These are the architectural components governing Compiere's architecture Performance & Scalability:
 
Volume:

Compiere handles transaction through its persistence layer which maps, secures and authenticates transaction consistency from dictionary meta data to database
The Compiere persistency engine is optimized for high volume OLTP environments.  For example, Compiere uses anonymous keys with the default settings of NUMBER(10) in the database and Java int. This restricts the system to 999 million records per table. If this is not sufficient, you can without code changes switch to Java Big Decimal and increase the scale of the ID column in the database to the maximum supported by the database.  Along with performance scalability, Compiere provides rigorous security and roles based access options. For example, the system can restrict users from creating queries with a defined number of records.  Long running processes can be executed in background or stared via scheduler thereby optimizing CPU performance.

Database:

Compiere uses a central database and is in production with Oracle Database Grids and database clusters.  Compiere manages transactions via a central service, eliminating the need for application developers to manage their transaction.  This increases uniformity and stability.  Compiere is available on Oracle and EnterpriseDB (PostgreSQL) with prototypes on DB/2 and Microsoft SQL Server.

Application Server:

Compiere can utilize any number of individual application servers, application server clusters and also Oracle Application Server Grids.  Each individual application server can be specialized for certain tasks or run in parallel to other application servers (e.g. multiple accounting servers). This allows for the IT infrastructure to scale up with the needs of the business application.  In addition to JBoss, Compiere was implemented with the Oracle Application Server and IBM WAS - and the web container also runs on stand alone Tomcat.

Payment Processors in Compiere

When you want to process credit card or electronic payments, you get a Merchant Account from your bank or other providers like Paypal. The Merchant account provider then may give you a choice of Payment Gateways, the companies who route the credit card requests to the issuers like Visa, Mastercard, American Express and manage the payments processing to your Merchant Account.

Compiere has provides an abstract class org.compiere.model.PaymentProcessor, which you can extend to interface to your specific payment gateway.  Compiere provides implementations for some of the the main gateways in the US - PayFlowPro (PP_PayFlowPro), Optimal Payments (PP_Optimal). I did also some research for interfacing to PayPal and Authorize.net.

You define the data required for the payment processor in the Window Bank - Tab Payment processor.  Here you define your payment processor class (e.g. org.compiere.model.PP_PayFlowPro) with your account/user names and passwords.

 

Document numbering

Compiere provides a variety of options how to generate and manage document numbers and formats. It also allows to maintain gapless sequence numbers. When any table has a the field "DocumentNo", it is recognized and managed as a document number. 

The default algorithm checks, if the field is empty and if so, a number is assigned. This number comes from an automatically created Document Sequence, which is sufficient for many areas. When a table has a Document Type (field "C_DocType_ID"), you determine for that Document Type if the numbering is manual the default behavior applies (i.e. if it is not entered, the number is created from the automatically created sequence). If the document type is document number controlled, a Sequence is referenced. Multiple document types can share a sequence or can have it's own sequence. 

A Document Sequence can also have a user defined prefix and suffix, so you can create document numbers like "Sales-12345-Export". Same algorithm here, if the Document number is empty, the number is created from the document sequence for the specific document type.