How Logging works with Log4j

In my last blog I explained the importance of logging in your enterprise application. In this post I am going to explain how we can used logging using Log4J logging framework.

 As always using a framework makes your life much easier and efficient. And main use of using a framework is it takes care of the underline platform. So when it comes to log4j which is an open source project created by Apache foundation and currently there are three logging frameworks

1.  Log4j for java 
2.   log4cxx for C++
3.   log4net for the Microsoft .NET framework.

So lets look at how log4j works...

 Log4j takes care of logging using three main components.  1) Loggers, 2) Appenders, 3) Layouts

  In simple terms if you want to know what these three main components does is mainly logger logs to an appender in a particular layout (pattern/style). But lets look at these three components in details

Loggers 

If you look it in the simplest way Loggers are actually logical class of file names.  These names are known to  your java application. For example if you have a class call HelloWorld.

The  standard logger should be 

Logger logger = Logger.getLogger("HelloWorld.class");

Logger should be defined class level. So one of the main advantage of using a framework for logging is you can control what you log in a hierarchical manner. If you put system.printline instead of using a logging platform you cannot actually disable some printlines while others are enabled. So here  using a logger you can actually define your log lines in an hierarchical order.

Loggers have 5 different hierarchical categories. I’l list them down by their order

 

 

Fatal  -  “The FATAL level designates very severe error events that will presumably lead the application to abort.” Fatal is the highest log level used to indicate that application is in a severe stage and application will get terminated due to this condition.

                            log.fatal(“Program termination”);

Error – “The ERROR level designates error events that might still allow the application to continue running.” Errors are not soo serious as Fatal  and program can still run with this condition however, it implies that program is going through an unexpected behavior in the execution flow. For example when the application is suppose to read a file and if that file not found you can log the exception using log.error(e.getMessage) to tell the user that this file is not there in the system.

Warn – warnings are used to tell the user that there is a chance of having a harmful situation due to a certain condition.  “The WARN level designates potentially harmful situations.”

Info – “ The INFO level designates informational messages that highlight the progress of the application at coarse-grained level.” Info is the most commonly used logger method. It is used to highlight the process of the event flow in an application.

Debug -   “The DEBUG Level designates fine-grained informational events that are most useful to debug an application.” This is used for debugging purposes if you need to debug your application you can enable debug level logs and see the execution flow.

Trace – “The TRACE Level designates finer-grained informational events than the DEBUG”  Gives more detail information than debug level. The lowest logger level. This is enable mainly to see finer grained information regarding the application.

Appenders

Another important feature of a logging API is to send logs to different locations. Depending on the user requirements logs should be sent to the console, remote monitor, o file systems. This is achieved by the appenders. Appender is responsible for the log destination.  There are couple of pre-define appenders in log4j.

 

1. ConsoleAppender - Sends log events to the System.out or System.err using a layout specified by the user. The default target is System.out
2. FileAppender - Sends log events to a file. (DailyRollingFileAppender, RollingFileAppender)
3. SoketAppender - Sends events to a remote log server, usually a SoketNode
4. JMSAppender - Sends events to a JMS topic.
5. NTEventLogAppender - Sends  events to the NT event log system.
6. SyslogAppender - Sends events to a remote syslog daemon.

And many more .. You can also define your own appender so that it will send log events to differant/multiple destnations. You can easily do it by extending AppenderSkeleton class. I will explain how that can be done in my next blogs.

Layouts

Layouts allow you to format your log messages before it is sent to the log destination. It actually stype your log message which can be very useful for filtering purposes and better monitoring approaches. By adding a pattern to your layout, you can exclude/include log event properties such as date time, log level, logger, message etc. You can also define your of layout by extending the log4j Layout class . Howeverm there are couple of predefine layouts in log4j such as PatternLayout, SimpleLayout, DateLayout, HTMLLayout, and XMLLayout.

Defining the layout in log4j.

log4j.appender.CARBON_LOGFILE.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout

log4j.appender.CARBON_LOGFILE.layout.ConversionPattern=TID: [%T] [%S] [%d] %P%5p {%c} - %x %m {%c}%n

log4j.appender.CARBON_LOGFILE.layout.TenantPattern=%U%@%D [%T] [%S]

Sample patternlayout using log4j

TID: [-1234] [Application Server] [2012-06-30 20:26:17,156]  INFO {org.wso2.carbon.coordination.core.services.impl.ZKCoordinationService} -  Coordination service disabled. {org.wso2.carbon.coordination.core.services.impl.ZKCoordinationService}

Monitor your application with system logs !!!

Tired of finding an issue with a current running application??? Your application started giving unexpected behavior and can’t figure out why? .. Cannot monitor your application properly? THEN LOGGING IS YOUR SOLUTION!!!!

 

Why Logging ???

Logging is a way of storing information about events happening during a program execution. Logging can be very useful when it comes to identify the event flow/ program execution or unexpected behavior of an execution such as errors/exceptions (error tracking) and it can be also useful when it comes to monitor performance.  Almost every developer use Logging!!! Even if they don’t use a special framework to do logging. For instance even if you are not using a particular logging frame work you most probably be using  system.printline in order to track  useful information and print them on the console so that you can monitor your application’s behavior and it’s current activities.

Importance of using a Logging Framework ?

However, using a logging frame work is a major plus point. Mostly because logs  are the very lifeline of your production application and it should not be taken lightly or as an afterthought. For example let’s say you don’t use logging at all and your application is out of development environment and your application is giving an unexpected behavior, it will be like finding a needle in a haystack to figure out what’s gone wrong in your application if you don’t use proper logging using a proper logging framework. 

  

Of cause using a logging framework is a plus point when it comes to enterprise applications. Because by using a logging frame work you can easily define log levels and filter logs accordingly. Further, when it comes to distributed applications (such as web/remote application) logging is a very crucial task because system administrators always needs to keep eye of what happening and  always need to see the logs for a given period of time. So filtering logs and viewing logs is also very important. This is why Logging has become the most frequent implementation of the Monitoring attribute and also in most cases it is the only implementation of monitoring (Mainly because it is the most easiest thing in the world for a busy developer to just put lnfor.log() in the code. However, apart from the ease of use logging frameworks also allow you to send logs to many different parties such as file systems, consoles, and databases. Which makes it easy for a system admin to filter logs from timestamp, log levels, and log messages.

Drawbacks of Logging

Even though, logging is a very important thing when it comes to developing application there is such thing  as too much logging and it’s important to know how to log efficiently.  Before we look into that lets see the common draw backs of bad logging.

Performance – There can be performance hit in your application if you don’t put the logs in the right way. One thing is if you have too many logs at unwanted places it can be a major performance hit and also it will reduce the speed of your application. And also pollute and increase the size of your code.

Scalability issues – When you log unwanted information or not have proper log levels in your application you might end up having huge log files. And it will increase the size of your log files. And it can eat up your software and hard ware resources.

Security – Logging can be security vulnerability because logs can carry security information such as passwords of user accounts and other sensitive information which may expose a system vulnerabilities.

These are very important points because our ultimate goal is to have efficient application which can be monitored efficiently at all time.

 

Best Practices of Logging

Therefore I would like to list down some of the best practices when it comes for logging however, I would like to discuss them in details in my next blog post.

1. Use an appropriate logging framework depending on your need - The proper selection of logging frame work will lead you to store and filter your information in an orderly fashion)
2. Proper use of Log levels - There are different levels of log levels choosing the right level at the right time can increase the performance and the scalability of your application.
3. Do Not Log unwanted information – Always make sure you know what you are logging and also make sure logging will not give side effects to your application. (Make sure you avoid null pointer exception while logging ;) ) And ALWAYS be concise and descriptive when you log 
4.  Make sure you use your own pattern. Logging frameworks let you design your own pattern layout which helps you filter your logs in an efficient manner. Make sure you use features of logging frame work when its needed.

How to Upgrade WSO2 ESB?

Following steps describes how to update(migrate) WSO2 products to newer versions

Step 1 - Deploying existing artifacts

Copy the deployed artifacts to WSO2 ESB old version to WSO2 ESB new version by copying wso2esb-4.0.0/repository/deployment/server folder to wso2esb-4.0.3/repository/deployment/server Also Copy all the content of repository/component/lib {wso2esb-4.0.0} to the new installation {wso2esb-4.0.3} (this will help you install the existing drivers and additional plug-ins used in your earlier installation)

Step 2 - Apply Patches

Most of the patches are applied when we release newer versions however, some patches can be not applied such as a custom patches or patches which were given after the release. Therefore you need to check the needed patches and apply them
Step 3 - Allocate proper memory numbers (if applicable).
If you have allocated memory numbers don't forget to allocate them to the new installation as well.

Step 3 - Change the configuration files

Apply the same changes you have done to configurations files inside OldVersion/repository/conf to NewVersion/repository/conf (if you have done any such to any configuration files)

Note

1. If you have done registry mounting make sure you apply to the new installation as done before by changing relevant configuration files such as carbon.xml,axis2.xml,user-mgt.xml,mgt-transports.xml
2. If you have created external data sources (Carbon data sources) make sure you copy datasources.properties file from repository/conf to the newer version repository conf

Step 4 - Apply security.

Apply the security measures taken as before. such as Encrypting passwords {as done previously}, Changing key stores etc

How to configure proxy service for an exsisting web application Using WSO2 ESB

norder to configure a proxy for an existing web application you need to set up binary relay, which allows users to send messages to differant parties at byte level while making decicions using transport headers. It further enables to passthrough SOAP messages without performing heavy XML parsing. Here are the steps to expose a webapplication through WSO2 ESB.

1. Enable the message relay module.
Go to ESB management console and go to Manage --> Modules --> List and click on engage icon associated with relay module to engage the module globally.

2. Configure message formatters and message builders in axis2.xml
Go to repository-->conf and edit the axis2 xml. Uncomment the nessary messageFormatters, messageBuilders as shown below.

<messageFormatters>
 <!--JSON Message Formatters-->
  <messageFormatter contentType="application/json"
          class="org.apache.axis2.json.JSONMessageFormatter"/>
  <messageFormatter contentType="application/json/badgerfish"

          class="org.apache.axis2.json.JSONBadgerfishMessageFormatter"/>

   <messageFormatter contentType="text/javascript"

          class="org.apache.axis2.json.JSONMessageFormatter"/>

    <messageFormatter contentType="application/x-www-form-urlencoded"

           class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

     <messageFormatter contentType="multipart/form-data"

            class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

     <messageFormatter contentType="application/xml"

             class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

     <messageFormatter contentType="text/html"
          class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

     <messageFormatter contentType="application/soap+xml"

           class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

     <messageFormatter contentType="text/xml"

            class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

      <messageFormatter contentType="x-application/hessian"

          class="org.apache.synapse.format.hessian.HessianMessageFormatter"/>

      <messageFormatter contentType=""
          class="org.apache.synapse.format.hessian.HessianMessageFormatter"/>

</messageFormatters>

<messageBuilders>

    <messageBuilder contentType="application/xml"

       class="org.apache.axis2.builder.ApplicationXMLBuilder"/>

    <messageBuilder contentType="application/x-www-form-urlencoded"

         class="org.apache.axis2.builder.XFormURLEncodedBuilder"/>

    <messageBuilder contentType="multipart/form-data"

          class="org.apache.axis2.builder.MultipartFormDataBuilder"/>

    <!--JSON Message Builders-->

     <messageBuilder contentType="application/json"

          class="org.apache.axis2.json.JSONOMBuilder"/>

     <messageBuilder contentType="application/json/badgerfish"

          class="org.apache.axis2.json.JSONBadgerfishOMBuilder"/>

     <messageBuilder contentType="text/javascript"

          class="org.apache.axis2.json.JSONOMBuilder"/>


     <messageBuilder contentType="application/xml"

          class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

     <messageBuilder contentType="application/x-www-form-urlencoded"

           class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

     <messageBuilder contentType="multipart/form-data"

           class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

     <messageBuilder contentType="multipart/related"

           class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

     <messageBuilder contentType="application/soap+xml"

           class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

      <messageBuilder contentType="text/plain"

           class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

      <messageBuilder contentType="text/xml"

            class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

      <messageBuilder contentType="x-application/hessian"

            class="org.apache.synapse.format.hessian.HessianMessageBuilder"/>

      <messageBuilder contentType=""

             class="org.apache.synapse.format.hessian.HessianMessageBuilder"/>

</messageBuilders>

Save axis2.xml and restart the server to affect the axis2.xml changes.


3.Creating the proxy service.


Go to ESB management console and  create a pass through proxy service by giving the target endpoint as your webapplication url.

<proxy xmlns="http://ws.apache.org/ns/synapse" name="amt" 

transports="http" statistics="disable" trace="enable" startOnLoad="true">

  <target>

     <outSequence>

        <send />

     </outSequence>

     <endpoint>

        <address uri="http://localhost/webapp/index.html" 

format="get" />

     </endpoint>

  </target>

</proxy> 

You can now access your proxy service through esb port. ie http://localhost:8282/services/MyProxyService1