Kartik Shah

SOA Composite deployment - Oracle SOA Suite

2011-07-09T13:15:00.000-05:00

This post is going to be about very specific subject - SOA Composite deployment on Oracle SOA Suite. I am going to capture few issues faced while deploying SOA composite.

Sample Composite

For purpose of this blog, consider a simple SOA composite that does the following:

Read entries from a database
Create XML from the entries
Store XML entries to another database
Read XML entries
Post each XML entry to a web service

There is good amount of workflow and business logic that makes it a perfect use case for doing it with BPEL, but that is beyond the purpose of this entry.

Composite Dependencies

Sample SOA composite had following dependencies:

Couple of Datasource (one to read source data and other to store XML Entries)
JMS Queue (Weblogic’s Uniform Distributed Queues)

Deployment Process

1. Datasources

Create Generic Datasources pointing to individual Oracle RAC node and a Multi Datasource using WLST scripts. (Most of the scripts are written for Weblogic 10.3.3, so haven’t explored newly introduced GridLinked Datasource in Weblogic 10.3.4.)

2. JMS Resources

Create JMS Server, JMS Module, Sub deployment, Connection Factory and JMS Queues using WLST scripts.

3. DBAdapter.rar

The composite uses DBAdapter connector to interact with databases, so add outbound connection pools the DBAdapter.rar pointing to actual Weblogic Datasources and redeploy application

4. Deploy Composite

Finally, deploy the composite using Enterprise Manager Console.

Lessons Learned

1. Server Start parameters

Using Enterprise Manager to deploy, the deployment window got stuck and never returned back. The issue, it turns out, is that the deployment was not able to communicate among instance servers. If SOA cluster is using multicast to communicate across weblogic instances, you will required following parameters on each server instance. The parameter adds well known address

-Dtangosol.coherence.wka1=soa01.mycompany.com -Dtangosol.coherence.wka2=soa02.mycompany.com -Dtangosol.coherence.localhost=soa01.mycompany.com -Xmx2048m

2. Failure updating DBAdapter.rar connector application

On attempt to create outbound connection pool pointing to weblogic datasource, activation fails complaining FileNotFoundException: Plan.xml. The file is present on the primary server instance, but it isn't replicated on other nodes. I am not sure why but for some reason the Plan.xml was required to present on all the nodes, even if deployment is being done from machine with AdminServer.

3. Failure on interaction with DB on some instances.

Even after the deployment was successful, any DB activities from nodes other than primary node failed giving following error:

BINDING.JCA-12511
JCA Binding Component connection issue.
JCA Binding Component is unable to create an outbound JCA (CCI) connection.
apptools:InsertLogDB [ InsertLogDB_ptt::insert(TestLogCollection,TestLogCollection) ] : The JCA Binding Component was unable to establish an outbound JCA CCI connection due to the following issue: BINDING.JCA-12510 JCA Resource Adapter location error.
Unable to locate the JCA Resource Adapter via .jca binding file element <connection-factory/> The JCA Binding Component is unable to startup the Resource Adapter specified in the <connection-factory/> element: location='eis/DB/application-ds'. The reason for this is most likely that either 1) the Resource Adapters RAR file has not been deployed successfully to the WebLogic Application server or 2) the '<jndi-name>' element in weblogic-ra.xml has not been set to eis/DB/application-ds. In the last case you will have to add a new WebLogic JCA connection factory (deploy a RAR). Please correct this and then restart the Application Server
Please make sure that the JCA connection factory and any dependent connection factories have been configured with a sufficient limit for max connections. Please also make sure that the physical connection to the backend EIS is available and the backend itself is accepting connections.

One noticeable thing on weblogic console was that under testing tab of DBAdapter.rar all outbound connection pools were not visible. The issue is that updated Plan.xml is not replicated to all other nodes. You have to manually copy the Plan.xml containing all outbound connection pool to all server instance nodes.

Here is what you need to do:

Copy the Plan.xml containing all outbound connection pool to all nodes
Update the connector application DBAdapter.rar
Restart DBAdapter.rar and Application server instance

4. BAM Cluster Multicast misconfiguration

A misconfiguration on the Multicast address of BAM cluster resulted in Access Forbidden 403. The issue was that BAM server was trying to communicate with another environment cluster. (i.e. prod bam cluster trying to communicate with test bam cluster).

Here is the error:

<BEA-000141> <TCP/IP socket failure occurred while fetching statedump over HTTP from 142374575950937656S:10.50.XX.XXX:[XXXXX,XXXXX,-1,-1,-1,-1,-1]:soa:soa_server1.
java.io.FileNotFoundException: Response: '403: Forbidden' for url: ' http://10.50.XXX.XXX:16101/bea_wls_cluster_internal/psquare/p2?senderNum=3&lastSeqNum=0&PeerInfo=10,3,4&ServerName=soa_server1'
        at weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:487)
        at weblogic.cluster.HTTPExecuteRequest.connect(HTTPExecuteRequest.java:67)
        at weblogic.cluster.HTTPExecuteRequest.run(HTTPExecuteRequest.java:83)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
   at weblogic.work.ExecuteThread.run(ExecuteThread.java:176)

So make sure that the Mutlicast address and port combination used for BAM cluster and SOA cluster is unique. There is a Multicast test utility that you can use to test communication between two instance. More information about the utility is available here. Also refer this to troubleshoot Multicast configuration.

export CLASSPATH=${CLASSPATH}:[bea_home]/server/lib/weblogic.jar

On Machine A:

java utils.MulticastTest -A [multicast address] -P [multicast port] -N TestServer1

On Machine B:
java utils.MulticastTest -A [multicast address] -P [multicast port] -N TestServer2

Hope this helps!.

OSB Project Structure

2011-04-20T13:13:00.001-05:00

I am doing some development with Oracle Service Bus (OSB). The primary goal is to mediate web services using OSB. Oracle does provide set of eclipse plugins to develop OSB configuration - OEPE suite. One of question that I wanted answer was what will be a good project structure to organize various components of OSB configuration.

Typically, OEPE plugins gives two eclipse project templates:

Oracle Service Bus Configuration Project
Oracle Service Bus Project

OSB Config project is the top level project which can include various OSB Project. A minimal OSB project includes components like business service, proxy service and WSDL. It is important to come up with a good project structure since the component shows up as is on sbconsole. You probably don't want to see all the components at same level.

After this exercise, I ended up with following structure of the project that I wanted to share. So here it is,

The top level project - OSB Config Project - represents a business domain. So OSB configuration of all services serving a particular domain goes under that umbrella. Individual OSB Project contains mediation component for a given physical service. I used folders to group business and proxy services to allow for multiple proxy services for a given business service. Using this structure it becomes easy to manage configuration using sbconsole.

I ended up using the ant build script and import/export WLST script by tweaking the ones given as part of this book - The definitive guide to SOA. The build script can export configuration from local server or workspace. Import deploys it to the target server. The import/export script also takes care of variables that changes from environment to environment. OSB has a features to create customization XMLs. You can create customization XML per environment and run it post configuration package deployment.

Using Clojure to work with RSS feeds

2011-01-23T23:18:00.001-06:00

As I mentioned in my last blog entry, I am trying to learn Clojure. I think it is an interesting challenge to internalize new thinking paradigm. So following entry/code is an attempt to learn the paradigms of the language. Drop a comment, if you have any suggestion.

I found that, while trying to learn new language, one of the challenge is to find interesting practice problem to solve. The practice problem which helps you understand the strength of the language. I think one such problem space Clojure's application would be interesting is parsing and data crunching.

Example Problem
So the problem I picked is a simple one. I heavily use google reader and for that matter most of my web reading happens as part subscribed RSS feeds. I find myself manually filtering the feed list to suite my interest at the time. This exercise creates parser for RSS feed. This parser creates Clojure records based data structure. Once you have all the feed represented in records structure you can write variety of functions to filter the feed or even apply intelligent web algorithms.

Organization of Code
I am looking for a good way to organize code in clojure. Just like Java packages, clojure has namespaces. But I have still not found a good way to organize clojure functions into files/namespaces. For now, I chose to create namespace per domain.

rss.data.clj - Define records data structure and parser
rss.feed.clj - Defines functions to get feeds
rss.stat.clj - Defines functions to filter feeds (and may be some statistical algorithms)

data.clj


(ns rss.data
  (:require [clojure.xml :as xml])
  (:import  [java.io ByteArrayInputStream]))

(defrecord rss-channel [title description link lastbuilddate pubdate items])
(defrecord rss-item [title description link guid pubdate])

(defn create-rss-channel [map items]
  (let [title (:title map)
        pubDate (:pubDate map)
        description (:description map)
        link (:link map)
        lastBuildDate (:lastBuildDate map)]
    (rss-channel. title description link lastBuildDate pubDate items)))

(defn create-item [item-data]
  (let [description (:description item-data)
        pubDate (:pubDate item-data)
        guid (:guid item-data)
        link (:link item-data)
        title (:title item-data)
        ]
    (rss-item. title description link guid pubDate)))

(defn parse-item [item-data]
  (create-item (zipmap (map :tag item-data) (map :content item-data))))

(defn parse-items [items-data]
  (map parse-item items-data))

(defn parse-channel [channel-content]
  (loop [x (:content channel-content)]
    (create-rss-channel
      (into {} (filter #(not (= :item (key %))) (zipmap (map :tag x) (map :content x))))
      (parse-items (map :content (into [] (filter #(= :item (:tag %)) x)))))
      ))

(defn parse-rss [rss]
  (let [rss-data (ByteArrayInputStream. (.getBytes rss "UTF-8"))]
  (for [x (xml-seq (xml/parse rss-data))
        :when (= :rss (:tag x))]
    (map parse-channel (:content x)))))

feed.clj

(ns rss.feed
  (:require [clj-http.client :as client]))

(defn get-feed [url]
  (:body (client/get url)))

(defn get-feeds [urllist]
  (pmap get-feed urllist))

stat.clj

(ns rss.stat)

(def keywords ["Clojure" "Groovy" "Python"])

(defn find-items [rss-item-list keyword]
  (filter #(.contains (first (:title %)) keyword) (flatten rss-item-list)))

(defn find-items-keywords [rss-item-list keyword-list]
  (flatten (map #(find-items rss-item-list %) keyword-list)))

Trying Delicious RSS
Let's use RSS feed published by delicious.com. All RSS feed can found here. Let's use the feed popular which got has tag "programming". (http://feeds.delicious.com/v2/rss/popular/programming)


user=> (rss.stat/find-items-keywords (map :items (flatten (rss.data/parse-rss (rss.feed/get-feed "http://feeds.delicious.com/v2/rss/popular/programming")))) ["Clojure" "Python" "Groovy"])
(#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#thaiyoshi"], :pubdate ["Sun, 23 Jan 2011 16:53:58 +0000"]} 
#:rss.data.rss-item{:title ["Writing Forwards Compatible Python Code | Armin Ronacher's Thoughts and Writings"], :description nil, :link ["http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/"], :guid ["http://www.delicious.com/url/c6fb2679bca6c193a41d92513f7edca1#papaeye"], :pubdate ["Sat, 22 Jan 2011 16:09:10 +0000"]})

This one only had few links. Let's try one which has lots of items. (http://feeds.delicious.com/v2/rss/tag/programming?count=100)


user=> (rss.stat/find-items-keywords (map :items (flatten (rss.data/parse-rss (rss.feed/get-feed "http://feeds.delicious.com/v2/rss/tag/programming?count=100")))) ["Clojure" "Python" "Groovy"])

(#:rss.data.rss-item{:title ["The Evolution of a Python Programmer : Aleks' Domain"], :description nil, :link ["http://metaleks.net/programming/the-evolution-of-a-python-programmer"], :guid ["http://www.delicious.com/url/15311aafa2207da0ffc770903091ccea#tsung"], :pubdate ["Mon, 24 Jan 2011 02:44:19 +0000"]} 
#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#adharmad"], :pubdate ["Mon, 24 Jan 2011 02:43:07 +0000"]} 
#:rss.data.rss-item{:title ["PySCeS: the Python Simulator for Cellular Systems"], :description nil, :link ["http://pysces.sourceforge.net/index.html"], :guid ["http://www.delicious.com/url/40e012b894d62cd8cb9ba6b95855b868#tiguco"], :pubdate ["Mon, 24 Jan 2011 02:39:24 +0000"]} 
#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#lonstile"], :pubdate ["Mon, 24 Jan 2011 02:24:25 +0000"]}

Source Code available at GitHub (https://github.com/kartikshah/rss-feeder)

Blogged with the Flock Browser

How-To - IntelliJ Idea for Clojure/Compojure on Google App Engine

2010-12-11T22:26:00.001-06:00

StrangeLoop conference really spiked my interest in learning Clojure. I wanted to find a way to do web development using Clojure. So this entry is primarily a how-to on setup of IntelliJ Idea to do Clojure using Compojure web framework, Leiningen build tool and Google App Engine deployment environment.

Step 1: Download Leiningen
Download Leiningen script file and follow instruction to put it in PATH.

Step 2: Install "La Clojure" and "Leiningen" plugin for IntelliJ Idea.
To do this go to Preferences -> Plugins. Go to "Available" Tab

Step 3: Setup Leiningen plugin properties
To setup Leiningen plugin properties in IntelliJ click on Preferences -> Leiningen and give the path to the executable.

Step 4: Create project template
Now with the installation complete, we are ready to create a new project. For now the project creation is done outside IntelliJ Idea using Lein on command line. This is the only step done outside the IntelliJ Idea. Choose the folder structure where your IntelliJ workspace is.


     $ lein new gae-cmpjr
     Created new project in: gae-cmpjr

You can see that Lein created project folder structure. In the project root it created a project.clj file used by Leiningen. Under src folder, it also created default namespace by the name of the project and created a core.clj file under it.

Step 5: Create project IntelliJ
Let's create IntelliJ Project, even though project structure exists pick the same folder path that you selected in step 4.

File -> Project New... and select project from scratch
Select both GAE and Clojure. Provide the path to the appengine sdk

Now basic setup for the project is done. You can browse the project files using IntelliJ. (Biggest benefit of La Clojure is it auto-inserts and matches brackets for very "brackety language" :-))

Step 6: Setup Leiningen to run from IntelliJ
To setup Leiningen from IntelliJ, click on Leiningen tool window. (Window -> Tool Windows -> Leiningen)

Step 7: Setup web development libraries
Setup the project to include Compojure and Ring libraries for development.
project.clj


(defproject gae-cmpjr "1.0.0-SNAPSHOT"
  :description "Example for Compojure on GAE using IntelliJ"
  :namespaces [gae-cmpjr.core] 
  :dependencies [[compojure "0.4.0-SNAPSHOT"]
                 [ring/ring-servlet "0.2.1"]]
  :dev-dependencies [[leiningen/lein-swank "1.2.0-SNAPSHOT"]]
  :compile-path "web/WEB-INF/classes"
  :library-path "web/WEB-INF/lib")

core.clj


(ns gae-cmpjr.core
  (:gen-class :extends javax.servlet.http.HttpServlet)
  (:use compojure.core
        ring.util.servlet)
  (:require [compojure.route :as route]))

(defroutes hello
  (GET "/" [] "<h1>Hello World Wide Web!</h1>")
  (route/not-found "Page not found"))

(defservice hello)

Step 8: GAE setup

web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
            http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
           version="2.5">
    <display-name>Compojure on GAE using IntelliJ</display-name>

    <servlet>
        <servlet-name>hello</servlet-name>
        <servlet-class>gae_cmpjr.core</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>hello</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>
</web-app>

appengine-web.xml


<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  <application>compgae</application>
  <version>v0-2</version>
</appengine-web-app>

Step 9: Compile and Build war

From the Leiningen Tool window in Idea run following commands.

lein clean
lein deps 
lein compile

Step 10: Run locally
To locally run click on already create "Run Configuration"

Step 11: To upload application to appengine
Tools -> Upload Appengine application

Blogged with the Flock Browser

Revisiting JMX DSL with Groovy GEP3

2010-09-25T23:11:00.001-05:00

In one of the previous blog post, I experimented with a sample DSL for JMX reporting using Jfree chart. Groovy 1.8.2 (Beta) has introduced command expression language. This GEP-3 essentially allows you to create command expression language which can further simplify language of DSL. The primary use of this is to aid in writing impromptu scripts to visually demo various MBean values. In this example we will see bar charts for module processing time for all web modules.

Command expression language allows you to alternate method name and parameter for even number of argument
e.g.
foo a1 a2 a3 a4 a5. Foo(a1).a2(a3).a4(a5)
You can find more example and explanation on GEP-3 page.

Revisiting JMX example so that I can simplify the language as


server.url "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" query "jboss.web:*" filter "j2eeType=WebModule"
def filteredModules = server.filteredModules

chart.type "Bar" modules filteredModules title "Module Processing Time" width 1200 height 700 refresh 500 attributes params labels graphLabels
chart.show()

Let's look at the supporting code.


import org.jfree.chart.ChartFactory
import groovy.swing.SwingBuilder

import org.jfree.data.category.DefaultCategoryDataset
import org.jfree.chart.plot.PlotOrientation as Orientation
import javax.swing.WindowConstants as WC
import javax.management.ObjectName
import javax.management.remote.JMXConnectorFactory
import javax.management.remote.JMXServiceURL as JmxUrl
import javax.naming.Context

class Chart {
    def chartModules
      def chartType
    def chartAttributes = {m -> [m.processingTime, m.path]}
    def chartLabels =["Time per Webapp", "Webapp", "Time"]
    def chartOptions = [false, true, true]
    def windowTitle
    def w
    def h
    def refreshRate
    def orientation = "VERTICAL"
    def dataset

    def modules(m) {
          chartModules = m
          this
      }

      def type(type) {
        chartType = type
         this
      }

      def attributes(attr) {
        chartAttributes = attr
        this
      }

      def labels (lbls) {
        chartLabels = lbls
        this
      }

      def options (opts) {
        chartOptions = opts
        this
      }

      def title (title) {
        windowTitle = title
        this
      }

      def width(width) {
        w = width
        this
      }

      def height(height) {
        h = height
        this
      }

      def refresh(r) {
        refreshRate = r
        this
      }

    void show(){
        switch(chartType){
            case "Bar": drawBarChart(); break;
            default: break;
        }
    }

    void drawBarChart(){
        calculateData()
        def chart = ChartFactory.createBarChart(*chartLabels, dataset, Orientation."${orientation}", *chartOptions)
        def swing = new SwingBuilder()
        def frame = swing.frame(title:windowTitle, defaultCloseOperation:WC.EXIT_ON_CLOSE){
            panel(id:'canvas') {rigidArea(width:w, height:h)}
        }
        while(true){
            calculateData()
            chart.fireChartChanged()
            frame.pack()
            frame.show()
            chart.draw(swing.canvas.graphics, swing.canvas.bounds)
            sleep(refreshRate)
        }
    }
    void calculateData(){
        def newDataset = new DefaultCategoryDataset()
        chartModules.each{ m ->
            def dsCall = chartAttributes.call(m)
            newDataset.addValue dsCall[0], 0, dsCall[1]
        }
        this.dataset = newDataset
    }
}


class Server {
    def server
    def queryObjects
    def moduleFilter
    def filteredModules

    def url(serverName){
    server = JMXConnectorFactory.connect(new JmxUrl(serverName)).MBeanServerConnection
    this
    }

    def query (queryString) {
        queryObjects = new ObjectName(queryString)
       this
    }

    def filter (filterString) {
         String[] allNames = server.queryNames(queryObjects, null)
        filteredModules = allNames.findAll { name -> name.contains(filterString)}.collect { new GroovyMBean(server, it)}
    }
}

server = new Server()
chart = new Chart()
params = {m -> [m.processingTime, m.path]}
graphLabels = ["Time per Webapp", "Webapp", "Time"]

server.url "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" query "jboss.web:*" filter "j2eeType=WebModule"
def filteredModules = server.filteredModules

chart.type "Bar" modules filteredModules title "Module Processing Time" width 1200 height 700 refresh 500 attributes params labels graphLabels
chart.show()

Problems faced
For some reason with the current beta build I couldn't do arrays or closures inline. so I had to put them as separate variables. I do not think it is by design and as 1.8.X comes close to release these wrinkles will be worked out.

Blogged with the Flock Browser

Data as Service - Thoughts

2010-03-31T23:46:00.001-05:00

I am exploring multiple ways to expose data as a service. Let's take a brief look at problem description. You have a central data repository which acts as source of truth for multiple supporting application. These supporting applications usually reads data and manipulate/transform it and do what it needs to do with it.

This is not very atypical scenario. In most of the organization you will have a central application - usually of the shelf product with specific customization. e.g. ERP systems, Billing systems, CRM systems etc. The data store supporting this application is not flexible to customization. You will also find supporting applications for custom needs of organization to use the central data store. In usual scenarios this application will be handling this via direct read from central data store - potentially giving way to duplication of effort to manage this objects.

These problems have been solved in multiple ways.

Allow each individual supporting application to access data directly or indirectly through data warehouses
Expose the catalog of finely-tuned queries expose by the supporting application as web services.
Define organization wide data directory containing well defined business entities/objects and expose web services to retrieve defined entities.

Types of Services
We can classify this data services in multiple categories

Data-Read Services - These are plain and simple data read or data lookup services. This services provide only minor transformation like column renames, etc
Transformation Services - These services provide some transformation of data. It operates on the column performs operation like truncation, concatenation, etc.
Filtering Services - These services allows opportunity to filter data based on input conditions. e.g provide active accounts, provide prospect accounts, etc.
Aggregation Services - These services provides performs certain aggregation function. e.g. sales by region, accounts by regions, etc. Though this logic could have served better purpose if kept in application, there are some performance benefits to do this in the data store layer.

Define Business Entities data dictionary
First and foremost identify organization wide data dictionary for the domain. This data dictionary is the definition of entities that applications needs to use. So there is not duplication. It also reduces confusion. For example, an entity named BillingAccount will mean the billing account for any application using it. This solves a larger problem, in the scenario where the supporting applications use the data directory it defines these central entities in multiple ways. So over time the definition of the same thing becomes convoluted. e.g. account number for application A is 10 digits and same for application b is first 8 digits of that. This results in duplication of data that can not inter operate. Defining a central data dictionary and allowing exchange of that in that contract helps avoid this problem.

Logical Representation
In most scenario the data store that comes bundled with the product is really generic so that product can be customized to different business. eg. ACCTPF having fields like ACCTPFNO, ACCTPFHONO, etc. One side effect can be seen on naming convention of entities and its properties. They are wierdly named and does not make any sense without actually looking at product documentation. My take is that some naming transformation is required. This can be done by "select as" queries or actually overlaying a logical data model on top of the existing one. Of course after performance consideration.

Expose as Web Service
Once you have the logical representation any of following two options can be followed

Write Select queries and expose them as either SOAP/REST web services
If select queries does not provide necessary transformation/aggregation, write db procedures and expose them as a service.

One of the sub variation of this scenario is to use DB specific data structures. (In case of Oracle, use oracle's user defined types). Performance consideration is essential because remember these services may very well be the most atomic activity overarching application will perform and it is essential that each data read is trivial and inexpensive.

Just some thoughts...

Blogged with the Flock Browser

Using Groovy Scriptom to report on Outlook

2010-03-16T23:05:00.001-05:00

As with most work places, I use MS Outlook Calendar at work for meetings, calendars, etc. With Outlook 2007 you have capability to categorize each meeting/time block that you have on your calendar. So you can classify each meeting as either design, architecture, learning, recurring, etc. Using Groovy Scriptom module you can do what you would usually do using VBScript. Scriptom allows you to use ActiveX windows components from Groovy. Needless to say this will only work on windows platform.

I used the scriptom to report on how I am spending my time. (ah.. have to fill those time sheet) Using Swingbuilder and JFreeChart, I threw in this simple script. You will need to setup dependency exactly as described on the help page here.

import org.codehaus.groovy.scriptom.ActiveXObject
import org.jfree.chart.ChartFactory 
import javax.swing.WindowConstants as WC

def outlook = new ActiveXObject("Outlook.Application")
def namespace = outlook.GetNamespace("MAPI")
def calFolder = namespace.GetDefaultFolder(9)
def myItems = calFolder.Items

def today = new Date()
startWeek = today - 7 
endWeek = today + 7
def categoryMap = [:]

for (i in 1..myItems.Count.value) {
     def currentMeeting = myItems.Item(i)
     if (currentMeeting.Start >= startWeek && currentMeeting.Start <= endWeek) {
           println "Subject: " + currentMeeting.Subject.value
           println "Category: " + currentMeeting.Categories
           println "Duration: " + currentMeeting.Duration.value

           category = currentMeeting.Categories
           durationValue = currentMeeting.Duration.value

           def value = categoryMap.get(category)
           value = value?value:0
           def newValue = value + durationValue
           categoryMap.put(category,newValue);
     }
}

def swing = new groovy.swing.SwingBuilder()
def titleString = 'Time Outlook: ' + String.format('%tm/%<td/%<tY', startWeek) + "-" + String.format('%tm/%<td/%<tY', endWeek)
def frame = swing.frame(title:titleString,
                        defaultCloseOperation:WC.EXIT_ON_CLOSE,
                        size:[800,600], locationRelativeTo: null) {
     borderLayout()
     panel(id:'canvas') { rigidArea(width:700, height:500) }
 }

def bounds = new java.awt.Rectangle(0,0, 700,500).bounds
def piedata = new org.jfree.data.general.DefaultPieDataset()
for(entry in categoryMap)
{
  piedata.setValue entry.key + " = " + entry.value, entry.value
}
def options = [false, true, false]
def chart = ChartFactory.createPieChart(titleString, piedata, *options)
chart.backgroundPaint = java.awt.Color.white
frame.pack()
frame.show()
chart.draw(swing.canvas.graphics, bounds)

Simple enough script... It gets the ActiveXObject for Outlook application. Then from namespace it gets calendarFolder (constant 9). You can find out more about constant here. calFolder contains all the meeting appointmentItems. Once you get to the appointmentItem you can ask for the properties using groovy dot-notation.

Here is the snapshot of the output

Blogged with the Flock Browser

Revisit Jmx Groovy DSL - Using AstTransformation

2010-02-28T23:59:00.001-06:00

In the previous blog entry, we created Jmx Charting DSL. Let visit some aspects of creating DSL using groovy.

In the previous blog example we created DSL using ExpandoMetaClass. Users will define the script starting with node jmx { }. With the use of ExpandoMetaClass we added the dynamic method which in turn delegated to class JmxClosureDelegate. Here is the snippet of the code.


static void runEngine(File dsl){
    Script dslScript = new GroovyShell().parse(dsl.text)
    dslScript.metaClass = createExpandoMetaClass(dslScript.class, {
      ExpandoMetaClass emc ->
        emc.jmx = {
          Closure cl ->
            cl.delegate = new JmxClosureDelegate()
            cl.resolveStrategy = Closure.DELEGATE_FIRST
            cl()
        }
    })
    dslScript.run()
   }

What we essentially did here:

Using GroovyShell it parses the script file passed as input.
Defines the ExpandoMetaClass and adds method by name "jmx" having closure as parameter.

So the script file you wrote gets a dynamic method injected into it using ExpandoMetaClass. This happens at runtime.

Getting rid of the Commas
The problem with creating methods with two arguments is that you have to specify commas between them. For example,


     server "nameofserver", {
          ...
     }

Undoubtedly commas clutter the langauge grammer. To get rid of the commas we used trick provided on Groovy users list.


//To avoid using "," between String and Closure argument  
  def methodMissing(String name, args) {  
    return [name, args[0]]  
  }

Using AstTransformation
Now lets explore a different possibility. You can achieve similar result using AstTransformation at compile time. The goal remains the same and that is to add method with name "jmx" with closure parameter.

First we will define the annotation.


//import statements skipped for brevity
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(["info.kartikshah.jmx.ast.JmxDslTransformation"])
public @interface UseJmxDsl {}

Transformation Class

This transformation class needs to perform two activities:

Add method jmx(Closure cl) method
Invoke the script method being defined

We need to generate AST statements for following snippet of code.


     jmx = {  
          Closure cl ->  
            cl.delegate = new JmxClosureDelegate()  
            cl.resolveStrategy = Closure.DELEGATE_FIRST  
            cl()  
        }

We will use AstBuilder's buildFromSpec option to generate it. (AstBuilders added with Groovy 1.7 definitely makes generating statement structure relatively easy and clutter free. Not to mention it is also an example of DSL added to the groovy language :-) )


@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)
class JmxDslTransformation implements ASTTransformation {
  static int PUBLIC = 1
  static int STATIC = 8

  void visit(ASTNode[] astNodes, SourceUnit sourceUnit) {
    //Add method jmx(Closure cl)
    ClassNode declaringClass = astNodes[1].declaringClass
    MethodNode jmxMethod = makeMethod()
    declaringClass.addMethod(jmxMethod)

    //Insert method call inside run method of the script class
    MethodNode annotatedMethod = astNodes[1]
    List<MethodNode> allMethods = sourceUnit.AST?.classes*.methods.flatten()
    MethodNode runMethod = allMethods.find{ MethodNode method ->
      method.name == "run"
    }
    List existingStatements = runMethod.getCode().getStatements()
    existingStatements.add(0, createMethodCall(annotatedMethod))
  }

  Statement createMethodCall(MethodNode methodNode){
    def statementAst = new AstBuilder().buildFromSpec {
      expression{
         methodCall {
           variable "this"
           constant methodNode.name
           argumentList {}
         }
      }
    }
    Statement stmt = statementAst[0]
    stmt
  }

  MethodNode makeMethod() {
    def ast = new AstBuilder().buildFromSpec {
      method('jmx', PUBLIC | STATIC, Void.TYPE) {
        parameters {
          parameter 'cl': Closure.class
        }
        exceptions {}
        block {
          expression {
            binary {
              property {
                variable "cl"
                constant "delegate"
              }
              token "="
              constructorCall(JmxClosureDelegate.class) {
                argumentList()
              }
            }
          }
          expression {
            binary {
              property {
                variable "cl"
                constant "resolveStrategy"
              }
              token "="
              property {
                classExpression Closure
                constant "DELEGATE_FIRST"
              }
            }
          }
          expression {
            methodCall {
              variable "cl"
              constant "call"
              argumentList {}
            }
          }
        }
      }
    }
    MethodNode jmxMethod = ast[0]
    jmxMethod
  }
}

Why use AstTransformation?
The question is why one would want to use AstTransformation when you can add method during runtime. For the given scenario, it is correct that you want to stick with adding method runtime. But consider scenario where you want to "redefine" meaning of Groovy's syntax. For example like following imaginary script using Statement Labels to add more readability to your DSL syntax.


@info.kartikshah.jmx.ast.UseJmxDsl
runDsl () {
  jmx {
    setup:
      server "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector"
      query "jboss.web:*"
      findAll "j2eeType=Servlet"

    draw:
    chart {
        chartType="Bar"
        attributes={m-> [m.loadTime, m.objectName.find("name=([^,]*)"){it[1]}]}
        labels=["Load Time per Servlet", "Servlet", "Time"]
        options=[false, true, true]
        windowTitle="JBoss Servlet Processing Time"
        width=1200
        height=700
        orientation="HORIZONTAL"
        refreshRate=5000
        show()
      }
  }
}

Spock Framework does similar twist by redefining meaning of existing construct.

With this type of language structure you will end up defining your own set of keywords, supporting parser and few AstTransformation to change the meaning of existing Groovy Syntax.

Blogged with the Flock Browser

A Groovy DSL - JMX Reporting

2010-02-16T22:30:00.001-06:00

In previous blog example, using JFreeChart, JMX and SwingBuilder we came up with dashboard type utility. With this post we will explore how to write small DSL which will allow to report on JMX using reporting charts. The DSL will using Groovy's SwingBuilder to draw JFree Chart to report on various MBeans. This type DSL can be used to write small scripts to monitor behavioral aspect of application server (or any JMX based application).

Simple DSL
First we will chart out how we want our domain language to look like. DSL syntax can be done multiple ways. It is necessary that you play with the syntax to come up with one that works for the scenario. For our example, here is the first cut that we will use.

Essentially, the script reports processing time of all web modules defined on (in this example - JBoss) application server using Bar Chart.


jmx {
    server "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" {
      query "jboss.web:*" {
        findAll "j2eeType=WebModule" {
          chart{
            chartType="Bar"
            attributes={m-> [m.processingTime, m.path]}
            labels=["Time per Webapp", "Webapp", "Time"]
            options=[false, true, true]
            windowTitle="JBoss Module Processing Time"
            width=1200
            height=700
            refreshRate=5000
            show()
          }
        }
      }
    }
  }

Other Use cases
Target users for this DSL are system administrators who can write simple scripts to monitor and/or report on various aspects of app server instances graphically.

Some other use cases for this type of DSL

Compare Processing Time of Web Application
Compare Load Time of Servlets
Compare Response Time
Compare memory usage
Compare total requests

DSL Engine
With Groovy there are multiple ways to write domain specific language as described in groovy documentation here

We will use nested closure approach. More information about nested closure here.

Here we use ExpandoMetaClass and series of Delegate classes to handle each closure. In the sample script above read each node (eg. jmx, server, query) as a dynamic method call with one or two arguments. For example jmx has a string argument followed by closure argument. String argument is we just store in instance variable. For closure argument we delegate the handling to separate Delegate Class. We follow the same pattern for the rest of the nested nodes.

First up Engine
This is the main class for the DSL. It gets the file passed as command args and pass it to GroovyShell to create Script object. It uses ExpandoMetaClass to dynamically add methods/closure. It add adds jmx closure and sets properties for handling closure.


class JmxReportingDslEngine {
  
  static main(String[] args){
    if(args.length != 1)
    {
      println("Usage: JmxReportingDslEngine <ScriptFileName>")
    }
    runEngine(new File(args[0]))
  }
  
  static void runEngine(File dsl){
    Script dslScript = new GroovyShell().parse(dsl.text)
    dslScript.metaClass = createExpandoMetaClass(dslScript.class, {
      ExpandoMetaClass emc ->
        emc.jmx = {
          Closure cl ->
            cl.delegate = new JmxClosureDelegate()
            cl.resolveStrategy = Closure.DELEGATE_FIRST
            cl()
        }
    })
    dslScript.run()
   }

  static ExpandoMetaClass createExpandoMetaClass(Class clazz, Closure cl){
    ExpandoMetaClass emc = new ExpandoMetaClass(clazz, false)
    cl(emc)
    emc.initialize()
    return emc
  }
}

Delegates

Next up we write series of Delegate Class responsible for handling each node of the language.

JMXClosureDelegate
This handles closure passed to jmxtag. It instantiates MBeanServerConnection and passes the reference down to the delegate chain.


class JmxClosureDelegate {
  //To avoid using "," between String and Closure argument
  def methodMissing(String name, args) {
    return [name, args[0]]
  }
  void server(param){
    def (serverUrl, cl) = param
    def server = JMXConnectorFactory.connect(new JmxUrl(serverUrl),env).MBeanServerConnection
    cl.delegate = new JmxServerClosureDelegate(server)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }

JMXServerClosureDelegate
This delegate handles closure passed to server tag.


class JmxServerClosureDelegate {
  def server
  JmxServerClosureDelegate(server){
    this.server = server
  }
  def methodMissing(String name, args) {
   return [name, args[0]]
  }
  void query(param){
    def (objectName, cl) = param
    def query = new ObjectName(objectName)
    String[] allNames = server.queryNames(query, null)
    cl.delegate = new JmxQueryClosureDelegate(allNames, server)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }
}

JmxQueryClosureDelegate
and so on...


class JmxQueryClosureDelegate {
  def allNames
  def server
  JmxQueryClosureDelegate(allNames, server){
    this.allNames = allNames
    this.server = server
  }
  def methodMissing(String name, args) {
    return [name, args[0]]
  }
  void findAll(param){
    def (filter, cl) = param
    def modules = allNames.findAll{ name ->
          name.contains(filter)
      }.collect{ new GroovyMBean(server, it) }
    cl.delegate = new JmxFindAllClosureDelegate(modules)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }
}

JmxFindAllClosureDelegate


class JmxFindAllClosureDelegate {
  def modules
  JmxFindAllClosureDelegate(modules){
    this.modules = modules
  }
  void chart(Closure cl){
    cl.delegate = new ChartDelegate(modules)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }
}

ChartDelegate
This class

Creates the dataset from the MBean values
Creates the chart with the dataset values
Creates the external frame using SwingBuilder


class ChartDelegate {

  def modules
  def chartType
  def attributes
  def labels
  def options
  def windowTitle
  def width
  def height
  def refreshRate
  def orientation = "VERTICAL"
  def dataset
  ChartDelegate(modules){
    this.modules = modules
  }
  void show(){
    switch(chartType){
      case "Bar": drawBarChart(); break;
     //TODO:Add more chart types
      default: break;
    }
  }
  void drawBarChart(){
    calculateData()
    def chart = ChartFactory.createBarChart(*labels, dataset, Orientation."${orientation}", *options)
    def swing = new SwingBuilder()
    def frame = swing.frame(title:windowTitle, defaultCloseOperation:WC.EXIT_ON_CLOSE){
      panel(id:'canvas') {rigidArea(width:width, height:height)}
    }
    while(true){
      calculateData()
      chart.fireChartChanged()
      frame.pack()
      frame.show()
      chart.draw(swing.canvas.graphics, swing.canvas.bounds)
      sleep(refreshRate)
    }
  }
  void calculateData(){
    def newDataset = new DefaultCategoryDataset()
    modules.each{ m ->
      def dsCall = attributes.call(m)
      newDataset.addValue dsCall[0], 0, dsCall[1]
    }
    this.dataset = newDataset
  }
}

Output
Let's run and look at sample output...

Click here for bigger images

Here is another script for Servlet Load Time

This one uses some intermediate groovy knowledge with attributes param taking a closure. It allows user to perform transformation on the MBean parameters. Below in the script it performs operates on long objectName attribute to get servlet name parameter.


jmx {
    server "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" {
      query "jboss.web:*" {
        findAll "j2eeType=Servlet" {
            chart{
              chartType="Bar"
              attributes={m-> [m.loadTime, m.objectName.find("name=([^,]*)"){it[1]}]}
              labels=["Load Time per Servlet", "Servlet", "Time"]
              options=[false, true, true]
              windowTitle="JBoss Servlet Processing Time"
              width=1200
              height=700
              orientation="HORIZONTAL"
              refreshRate=5000
              show()
            }
        }
      }
    }
}

and here is the output...

Click here for bigger image

Further you can,

Add different type of chart support to the DSL - Bar, XY, trending, etc
Use/Extend to work with different application server and/or application

Complete source code can be found at http://github.com/kartikshah/jmx-dsl

Blogged with the Flock Browser

Exploring Google Collections - Part 2

2010-02-05T21:35:00.001-06:00

In Part 1, we focused on few classes of google collection. Here in part 2, we will expand on the simple scenario and work on DOM like tree structure scenario.

Consider a scenario of an external API provided by vendor. The API had tree like data structure, described by diagram below.

Now the problem was that the main data structure part of external API was POJO. Though it was a tree structure it did not provide any operations for traversal or find operations. Here is the watered down version of the component.

public class NodeComponent {
    private String name;
    private List<NodeComponent> children;
    private Map<String, String> attributes;

    public NodeComponent(String name)
    {
        this.name = name;
    }

    // Getters and Setters omitted for brevity
}

Let's see how google collection can be used to provide nice searcher methods. Consider following use cases:

Find child nodes based on node name
Find child nodes based on attribute name and value
Find child nodes on composite criteria of node name and attribute name and value

Lets implement cases using google collections.

Case 1: Find child nodes based on node name
First we will define NodeNameCriteria implementing Predicate interface


public class NodeNameCriteria implements Predicate<NodeComponent>
{
    private String nameCriteria;

    public NodeNameCriteria(String nameCriteria)
    {
        this.nameCriteria = nameCriteria;
    }

    public boolean apply(NodeComponent node) 
    {
        return node.getName().equals(nameCriteria);
    }
}

Further, define NodeSearcher class which will represent current node and will provide all these helper methods


public class NodeSearcher
{
    private NodeComponent currentNode;

    public NodeSearcher(NodeComponent currentNode)
    {
        this.currentNode = currentNode;
    }

    public Collection<NodeComponent> findChildrenByNodeName(String name)
    {
        return Collections2.filter(currentNode.getChildren(),new NodeNameCriteria(name));
    }
}

Case 2: Find child nodes based on attribute name and value
Similar to first approach, this requires creating a Predicate implementation, let's name it AttributeNameValueCriteria


public class AttributeNameValueCriteria implements Predicate<NodeComponent>
{
    private String nameCriteria;
    private String valueCriteria;

    public AttributeNameValueCriteria(String nameCriteria, String valueCriteria)
    {
        this.nameCriteria = nameCriteria;
        this.valueCriteria = valueCriteria;
    }

    public boolean apply(NodeComponent component)
    {
        Set<Map.Entry<String, String>> entrySet = component.getAttributes().entrySet();

        Set<Map.Entry<String, String>> matchedAttrSet = Sets.filter(entrySet, new Predicate<Map.Entry<String, String>>(){
            public boolean apply(Map.Entry<String, String> entry) {
                return nameCriteria.equals(entry.getKey()) &&
                    valueCriteria.equals(entry.getValue());
            }
        });
        return matchedAttrSet != null && !matchedAttrSet.isEmpty();
    }
}

Case 3: Find child nodes on composite criteria of node name and attribute name and value
So far so good but the real benefit of defining the Predicate comes with the third case. In this case what is needed is to define a criteria which is composition of NodeNameCriteria and AttributeNameValueCriteria. We don't need to define third criteria implementation. Instead we will use Predicates compose method. Add following implmentation of findChildrenByNodeNameAndAttributeNameValue to NodeSearcher

public Collection<NodeComponent> findChildrenByNodeNameAndAttributeNameValue(String nodeName, String attrName, String attrValue)
{
     Predicate<NodeComponent> compositePredicate = Predicates.and(new NodeNameCriteria(nodeName),
                                                        new AttributeNameValueCriteria(attrName, attrValue));
     return Collections2.filter(currentNode.getChildren(), compositePredicate);
}

In simpler approach, one would have defined filter criteria as condition of if loop. But with implementing them as Predicate implementation, you can reuse criteria implementation and mix and match with help of following Predicates function and and or methods.

Blogged with the Flock Browser

Exploring google collections - Part 1

2010-01-31T22:49:00.001-06:00

Google Collection 1.0 released
Google collections finally hit version 1.0. API provides rich data structures building on JDKs collection API. It also provides utility classes for working with collections. With this post I want to focus on few google collections classes and demonstrate how they help in working with collections. In part 1 of the post, we will deal with very simple example. Later in part 2 of the post, we will work with more involved real life example.

We will focus on these classes:

Scenarios
While working with collections you may have noticed that code gets cluttered with for loops, nested for loops, or with for loops with embedded if loops. In most of these scenario what you are trying to do is to iterate through a given collection, either to search for specific elements or to operate on specific elements.

Lets break down the typical cases:
1. Iterating through a given collection and finding a specific element(s) based on criteria


for(Element element: elementsCollection){
     if(element.equals(something){ //Or any other test criteria based on state of element object
          //either add to collection of elements
          //or return this element
     }
}

2. Iterating though a given collection and if certain criteria matches perform operation on the collection


for(Element element: elementCollection){
     if(element.equals(something){
          //or operate on this element
     }
}

As an outcome of this, code ends up with

Code duplication of your iteration logic
Code duplication of your condition (matching criteria)
Code duplication of operation on the specific elements (This however can be avoided, if operation is re-factored into a method)

Google Collections Approach
Lets start with very simple example, here is how google collection can help:
1. Define you matching criteria using Predicate like


            Predicate<String> validEmailAddress = new Predicate<String>(){
                public boolean apply(String emailAddress){
                    return emailAddress.contains("@");
                }
            };

This can be used as


List<String> emailList = Lists.newArrayList("abc@gmail.com", "xyz@gmail.com", "aaagmail.com");
Collection<String> validEmailList = Collections2.filter(emailList, validEmailAddress);

2. Define your operation on element using Function like


        Function<String, String> extractUserName = new Function<String, String>(){
            public String apply(String emailAddress)
            {
                String userName = emailAddress.substring(0, emailAddress.indexOf('@'));
                return userName;
            }
        };

This can be used as


        List<String> emailList = Lists.newArrayList("abc@gmail.com", "xyz@gmail.com", "aaa@gmail.com");
        List<String> userNameList = Lists.transform(emailList,extractUserName);

We will take on more complex DOM-like tree based data structure in part 2 of the post...

Blogged with the Flock Browser

Helpful tips - Working with AST Transformation

2010-01-23T10:19:00.001-06:00

I have been playing with Groovy AST transformation for some time now. You can walk through some of the earlier posts here. With this entry I want to capture some of the lessons learnt, tips, etc.

Using AstViewer Tool (Inspect Ast)
With Groovy 1.6 AST transformation, one of the difficulty was to come up with tree structure for the code you want to inject or change. It involves calling complex set of Statement and Expression APIs. It is ridden with intricate details of language grammar. With groovy 1.7 AstBuilder, you can use buildFromString and buildFromCode which is lot more easier. But still if you have to use buildFromSpec you will need to have some understanding of Statement/Expression API.

One of the tool I have found useful is Groovy Console's AST Browser or Eclipse's ASTViewer. Write code you want to generate language structure for and try to replicate using API or using AstBuilder's buildFromSpec. The tool has gotten better from 1.6 to 1.7.
Open Groovy Console and type in following code


            def advice = new String("Hello AST")
            println advice

Go to Script --> Inspect Ast (or press Ctrl+T) and you will see the AST structure needed.

Educated Guess in Script
With AstBuilder now you can just start building AST in main method or just as script. This has proven to be useful if you just want to compare what you are generating with what is displayed in AstBuilder.


def nodes = new AstBuilder().buildFromXXX{
}

Working with Variables
If you are using variables that gets replaced during runtime you may face problem generating AST. One of the method that I found useful was to replace the variable with static value and see if it works. If it does than the problem is possibly due to the type of the variable.
For example, I wanted to generate


          def advice = new <<variableClassName>>()

where variableClassName will be passed in from annotation value. One option is to try generating it with static value like java.lang.String (or just String), if it works than there is some problem with the type of the variable you are passing.

Compile file individually (if using IDE)
Sometimes the tools you are working with just does not seem to give different output even when you are changing things around in your code. Sometimes stale classes just gets called and you never see the output from the updated classes. You may want to compile both transformation class and the class where the transformation gets used.

ByteCode Viewer
Also using Byte Code Viewer like this helps in looking at the classes that are getting generated. You can also write your class how it will look after AST transformation. Use byte code viewer to see the class and compare it with the class that is being generated through AST Transformation. See which statements are properly generated and which are not.

Testcases
Another extremely resourceful source for AstBuilder is test cases found here. It contains rich example set of how to use each node in the builder specifically for buildFromSpec option. AstAssert is another excellent utility to compare and test expected and actually generated structure. Looks like AstAssert class is not packaged with groovy-1.7, so you may need to download and drop it in your project.

Divide and Conquer
Treat each section of code for which AST is being generated as separate section. Comment out blocks of code and gradually uncomment them to see which section is actually failing and/or not giving desired output.

Hope this helps! Drop in a line to share your tips or problems you face with Groovy's Ast transformation.

Blogged with the Flock Browser

Revisiting Groovy AST Transformation with AstBuilder

2010-01-18T00:37:00.001-06:00

Groovy AST is powerful tool to inject changes into classes at various compile phases. It allows one to operate on AST (Abstract Syntax Tree) to manipulate class structure.

With Groovy 1.7 AstBuilder there are easier ways to write AST transformation. One can build AST nodes using one of following three options

Build From String
Build From Spec
Build from Code

Previously on this blog, we experimented with Groovy 1.6 AST Transformation Example and also AOP style usage at Groovy AST Transformation - AOP Style. Lets revisit this transformation from earlier post.

Lets see how we can generate the AST tree using these new methods from AstBuilder. Recap this previous post before proceeding ahead.

Looks like we need to rewrite initAdviceCall and createMethodCall to experiment with new methods of AstBuilder.

Build From String
Using buildFromString the task would become very easy. With this options, it is not required to build complicated statement and expression structure. All you need to do is embed code within string and pass it in to buildFromString method. If code you generate is not completely static in nature, you may have to parameterize the value. Here in this example, the declaration of "advice" instantiation depends on the Class value defined in the annotation.


  def initAdviceCall(classNodeType) {
    def codeString = "def advice = new " + classNodeType.type.typeClass.canonicalName + "()"
    def buildNodes = new AstBuilder().buildFromString(CompilePhase.SEMANTIC_ANALYSIS, true, codeString)
    Statement stmt = buildNodes[0].statements[0]
    stmt
  }

  def createMethodCall(method, Parameter[] parameters) {
    def codeString = "advice.before (" + "'$method.name'" +", [" + parameters.collect {it.name}.join(',') + "])"
    def buildNodes = new AstBuilder().buildFromString(CompilePhase.SEMANTIC_ANALYSIS, true, codeString)
    Statement stmt = buildNodes[0].statements[0]
    stmt
  }

Build From Spec
buildFromSpec option is more verbose. It also requires you to understand internal structure that is required to be build. With use of GroovyConsole's Inspect Ast option, it is not difficult to come up with the AST structure.


  def initAdviceCall (classNodeType) {
    def buildNodes = new AstBuilder().buildFromSpec {
      expression {
        declaration {
          variable "advice"
          token "="
          constructorCall(classNodeType.getType().getTypeClass()){
            argumentList()
          }
        }
      }
    }
    Statement stmt = buildNodes[0]
    stmt
  }

  def createMethodCall(method, Parameter[] parameters) {
    def buildNodes = new AstBuilder().buildFromSpec {
      expression {
        methodCall {
          variable "advice"
          constant "before"
          argumentList {
            constant method.name
            list {
              parameters.each {variable it.name}
            }
          }
        }
      }
    }
    Statement stmt = buildNodes[0]
    stmt
  }

There are some differences between using API directly and using spec builder option. One thing specifically that threw me off was that ConstructorCallExpression takes in ClassNode as parameter, however the constructorCall in builder takes Class. It wouldn't have been much of an annoyance, if I had read GEP-2 which states that any expression taking ClassNode, for builder you just need to pass class instance. Well in this example, value provided by annotation was ClassNode, so I had to get Class and pass it on to constructorCall.

Build From Code
Unlike above two options buildFromCode option did not work for me in above scenario. The problem I faced was I couldn't do def advice = <VariableClassName>() without using reflection. Method would have looked something like this


def initAdviceCall(classNodeType){
new AstBuilder().buildFromCode {
     def advice = classNodeType.getType().getTypeClass().newInstance()
}

Similarly, for createMethodCall, I would have to use reflection for passing in parameters.

More later...

Blogged with the Flock Browser

Groovy Truth

2010-01-05T07:48:00.001-06:00

Groovy had its truth. Groovy expanded on what Java allows to use in various control structures. While Java allows only boolean expression in if and while control structures, Groovy relaxed the requirement. In Groovy you can use objects in if and while expressions. Only a non-null and non-empty string will evaluate to true. A non-empty collection will evaluate to true. Non zero numbers will evaluate to true.

With Groovy 1.7, Groovy Truth got better. You can now customize the Truth based on your requirement. This can be done by implementing boolean asBoolean() method. Implement the method on any object and logic defined in the method will be used to evaluate. Look at the following example


class Account {
     ...
     String status
     boolean asBoolean(){
          if(status != 'active') 
               false
          else 
               true
     }
}
assert new Account(status:'active')
assert !new Account(status:'prospect')

You can also expand (or change) existing classes like String, Integer, etc using metaClass. Let's expand the groovy truth to return null if the string value is 'null'


     String.metaClass.asBoolean = {
                                    if(delegate  == null || delegate == '' || delegate == 'null') false
                                    else true
                                  }

     String nullString = null
     String nullLiteralString = "null"
     String validString = "JohnDoe" 
     String emptyString = ''
     
     assert !nullString
     assert !nullLiteralString 
     assert validString
     assert !emptyString

Asserts are one place where the overridden logic will be applied. It works with if,while, and ternary operator as well.


//continued from previous script
println "null"?"FALSE":"TRUE"

str1 = "null"
if(str1) println "Valid String" 
else println 'Null, empty or "null" literal string'

If you do not want to override the way an object evaluates to boolean globally. You can use categories.


class StringNullTruthCategory{
    boolean asBoolean(String str){
        if(str || str == 'null')  false
        else true
    }
}

use(StringNullTruthCategory){
   String parameterValue = "null"
   if(parameterValue)   {
       println 'Should not be printed'
   } else {
       println 'Should print'
   }
}

def httpRequestParmValue = "null"
print 'Valid Value: '
println httpRequestParmValue?'YES':'NO'

Similarly, you can expand Integer's truth value if value is either zero or negative value. you can expand truth value of Map based on specific key-value. You have to be careful in doing so though, completely overriding the evaluation criteria can lead to confusion. You can apply on customize builder object. Let's say you have customized builder parsing order.xml into OrderEntity who has multiple OrderItems. OrderEntity truth can be derived from all of its OrderItem.


class OrderEntity{ 
     ...
     OrderItem[] orderItems
     String status
     ...
     boolean asBoolean(){
          //Define based on status or status of orderItems or presense/absence of orderItems
     }
}

if(currentOrder) { 
     //do time consuming process
}

You can define asBoolean objects in your Java class and use that definition to determine boolean value. However, you have to be careful of legacy Java Objects which has method asBoolean already defined for different purpose.

Blogged with the Flock Browser

Mule Notes

2009-12-30T21:43:00.001-06:00

Mule Mind Map

Click Here for full size image

Blogged with the Flock Browser

Grails: belongsTo

2009-11-19T23:24:00.001-06:00

In Grails hasMany and belongsTo are used to define relationship between objects. There are various other uses of belongsTo. Trying to capture them here in this short post.

Let's take an example, say you have domain class TaskList which had many Task

Direction of Access
In class Task if you define belongsTo as
belongsTo = TaskList
it means that Task object can not access its associated instance of TaskList
If it is defined as
belongsTo = [taskList: TaskList]
it means that Task object can access its associated instance of TaskList by doing task.taskList

Cascading Deletes
Only if belongsTo is defined cascading delete will happen. If TaskList is deleted all Tasks part of that TaskList will be deleted.

addTo* method access
For many-to-many relationship, defining belongsTo enables dynamic addTo* method
Extending previous example, say each Task can have multiple Tag and a Tag can be given to multiple Tasks. Defining belongsTo enables addTo* dynamic methods. e.g. task.addToTags(tag)

Blogged with the Flock Browser

Groovy AST Transformation - AOP Style

2009-07-10T00:25:00.001-05:00

In my last blog about AST Transformation I followed a sample example and created AssertParamsNotNull local transformation. Further, I wanted to create AST Transformation which is generic enough and performs any check before method is executed. In process, goal is to learn about AST Transformation.

BeforeAdvisor AST Transformation
Use case for BeforeAdvisor AST involves:

Authorization Checking - Security by checking role from context
Print Parameter values with which the method is called
Asserts Parameters are not null
Check various entry-conditions/Pre-Conditions of the method

To make transform generic enough, idea is to inject a method call to before method of advice for each method annotated with @BeforeAdvisor(MyPreConditionAdvice). Advice method can choose to implement any checks/conditions to be performed before actual methods gets executed.

Few of the subtle characteristics of this type of solution are

It does not allow changing the method parameters.
It does not allow to stop execution of method. However, you can throw runtime exception.
Advice needs no arg constructor and must implement method before

Usage and Client
Starting with class that will use this transform. Following defines script-level method with annotation having advice information.


package com.learn.sts.groovy.ast

@com.learn.sts.groovy.ast.BeforeAdvisor(value= com.learn.sts.groovy.MyAdvice)
def sayHello(name, name2)
{
    println "Hello " + name + name2
}

sayHello("World", "Groovy")

Sample Advice
The advice that we will try to invoke will be something like this. This advice just prints parameter value that method is being invoked with. But you can implement any of the use cases described above.


package com.learn.sts.groovy

public class MyAdvice
{
    def before(String methodName, List listArg)
    {
        println 'Entering Method ' + methodName + ' with params ' + listArg
    }
}

Annotation
Now lets create the Annotation


package com.learn.sts.groovy.ast;

import java.lang.annotation.Retention
import java.lang.annotation.RetentionPolicy
import java.lang.annotation.Target
import java.lang.annotation.ElementType
import org.codehaus.groovy.transform.GroovyASTTransformationClass
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(["com.learn.sts.groovy.ast.BeforeAdvisorASTTransformation"])
public @interface BeforeAdvisor {
    Class value ();
}

One difference to this annotation is that it declares a method value(). This allows us to get value being passed during annotation declaration on method.

AST Transformation


package com.learn.sts.groovy.ast

//Imports section skipped for brevity

@GroovyASTTransformation(phase=CompilePhase.SEMANTIC_ANALYSIS)
public class BeforeAdvisorASTTransformation implements ASTTransformation 
{
    public void visit(ASTNode[] nodes, SourceUnit source)
    {
        AnnotationNode node = (AnnotationNode) nodes[0];
        final Expression classNode = node.getMember("value")
        
        List<MethodNode> allMethods = source.AST?.classes*.methods.flatten()
        List annotatedMethods = allMethods.findAll{ MethodNode method ->
                 method.getAnnotations(new ClassNode(BeforeAdvisor))
                }
        annotatedMethods.each{MethodNode method -> 
             List existingStatements = method.getCode().getStatements()
            Parameter[] parameters = method.getParameters()
            int i = 0;
            existingStatements.add(i++, initAdviceCall(classNode))
            existingStatements.add(i++, createMethodCall(method, parameters))
        }
      }
  
    public Statement initAdviceCall(classNode)
    {
      return new ExpressionStatement(
              new DeclarationExpression(
                      new VariableExpression("advice"),
                      new Token(Types.ASSIGNMENT_OPERATOR, "=", -1, -1),
                      new ConstructorCallExpression(classNode.getType(), new ArgumentListExpression())
                      )
              )
    }
 
  public Statement createMethodCall(method, Parameter[] parameters){
      List parameterExpressionList = new ArrayList()
      parameters.each{ parameter -> parameterExpressionList.add(new VariableExpression(parameter))}
      return new ExpressionStatement(
        new MethodCallExpression(
            new VariableExpression("advice"),
            "before",
            new ArgumentListExpression(new ConstantExpression(method.getName()),
                    new ListExpression(parameterExpressionList)
            )
        )
      )
  }
}

Couple of things to notice here:

First, creation of the AST tree involves

Creating instance of Advice Class - This is done through method initAdviceCall
Invocation of the method before with methodName and Parameter List - This is done through method createMethodCall

Creating AST structure is not the easiest task. One approach that has worked for me is the write the sample code that you are trying to generate AST for and then use Groovy AST viewer in eclipse.

Second, inspecting the value on ASTNode and getting Advice class value. First node contains information about the annotation and second node is the annotated node.

Lessons Learned

Compilation
It was required to compile Transformations files first and then compile the classes that use AST Transformation. If I compiled all three of them together the AST Transformation did not kick in. I used Eclipse IDE and it forced me to use Compile Groovy File explicitly each time I made change. "Build Automatically" or "Clean" option did not work.

Script Level Methods vs all Class Methods
In AssertParamsNotNull AST Transformation in previous blog, I used source.getAST()?.getMethods() what it did is that it only found top level (Script Level) methods. So AST Transformation did not apply to Class Methods, it only got applied to Script Level Method. For this one I changed to source.ast?.classes*.methods.flatten() (Line 12 in BeforeAdvisorASTTransformation). This also become apparent once you see AST structure for a given class using Groovy AST View in eclipse.

Retrive value from annotations
ASTNode being passed to visit method carries information about the annotation. First element (node[0]) contains information about annotation. You can get the values passed in to annotation during the visit method and use it during the transformation. In case above we get class passed in as value and it gets instantiated and before method gets invoked.

Groovy Compilation
When groovy compiler is invoked, any sourcefile.groovy goes under series of transformations.
From Source --> ANTLR Tokens --> ANTLR AST --> Groovy AST --> Bytecode

Using AST Transformation, we manipulate the way groovy AST gets generated. It allows to insert additonal statements.

There are various CompilerPhase that goes along with this process. I couldn't find much documentation on the process. Best information is found on Jochen Theodorou's blog post

Blogged with the Flock Browser

JMX, Groovy, JFreeChart and Swing

2009-06-20T23:35:00.001-05:00

Recently, I worked on creating JMX based dashboard application showing Weblogic instances' information. During the exercise I came across JMX and Groovy examples

Using JMX, Groovy, Swing and JFreeChart, I came up with a dashboard application which looks like this.

It show memory usage on two instances. It automatically updates every 10 seconds and redraws chart.

Surprisingly (or not so surprisingly), code required to be written to perform this activity is minimal. Specifically thanks to Groovy's SwingBuilder and JMX libraries. Groovy makes the task of using 4 different technologies really easy and fun.


import org.jfree.chart.ChartFactory
import javax.swing.WindowConstants as WC
import javax.management.remote.*
import javax.naming.Context;

/**
 * WeblogicJMXReporter monitors Memory value for given node
 * @author kartik.shah
 */
public class WeblogicJMXReporter
{

    static void main(String[] args)
    {
    	def env = [:]
    	env["java.naming.factory.initial"] = "weblogic.jndi.WLInitialContextFactory"
    	env[Context.SECURITY_PRINCIPAL] = "system"
    	env[Context.SECURITY_CREDENTIALS] = "*******" //Password is always stars
    	env[JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES] = "weblogic.management.remote"
    	def jmxServiceURL = new JMXServiceURL("http", "10.50.120.110", 19400,"/jndi/weblogic.management.mbeanservers.domainruntime")
    	def server = JMXConnectorFactory.connect(jmxServiceURL, env).MBeanServerConnection

    	def chart1Map = getChart("node1_svr", server);
    	def piedata1 = chart1Map."pieData"
    	def chart1 = chart1Map."Chart"
    	def jvmInfo1 = chart1Map."MBean"
    	
    	def chart2Map = getChart("node2_svr", server);
    	def piedata2 = chart2Map."pieData"
    	def chart2 = chart2Map."Chart"
    	def jvmInfo2 = chart2Map."MBean"
    	
    	def swing = new groovy.swing.SwingBuilder()

    	def frame = swing.frame(title:'Weblogic INT Memory Usage', defaultCloseOperation:WC.EXIT_ON_CLOSE, 
    	                        size:[800,600], locationRelativeTo: null) {
    	    borderLayout()
    	    panel(id:'canvas') { rigidArea(width:700, height:250) }
    	}

    	def bounds1 = new java.awt.Rectangle(0,0, 350,250).bounds
    	def bounds2 = new java.awt.Rectangle(351,0, 350, 250).bounds
    	while(true)
    	{
    	    recalculatePieData(piedata1, jvmInfo1)
    	    recalculatePieData(piedata2, jvmInfo2)
        
        	chart1.fireChartChanged()
        	chart2.fireChartChanged()
        	
        	frame.pack()
        	frame.show()
        	chart1.draw(swing.canvas.graphics, bounds1)
        	chart2.draw(swing.canvas.graphics, bounds2)
        	sleep(10000)
    	}
    }
    
    static getChart(nodeName, server)
    {
    	def jvmInfo = new GroovyMBean(server, 'com.bea:Location='+nodeName+',Name='+nodeName+',ServerRuntime='+nodeName+',Type=JVMRuntime')
    	def piedata = new org.jfree.data.general.DefaultPieDataset()
    	piedata.setValue "Free", jvmInfo.HeapFreeCurrent
    	piedata.setValue "Used", jvmInfo.HeapSizeMax - jvmInfo.HeapFreeCurrent
    	def options = [true, true, true]
    	def chart = ChartFactory.createPieChart(nodeName, piedata, *options)
    	chart.backgroundPaint = java.awt.Color.white
        
        ["MBean":jvmInfo, "pieData":piedata, "Chart":chart]
    }
    
    static recalculatePieData(piedata, jvmInfo)
    {
    	piedata.setValue "Free", jvmInfo.HeapFreeCurrent
    	piedata.setValue "Used", jvmInfo.HeapSizeMax - jvmInfo.HeapFreeCurrent
    	println piedata.getValue("Free")
    	println piedata.getValue("Used")

    }
}

For purpose of this blog and keeping it simple, I kept it all in one groovy class.

Blogged with the Flock Browser

Groovy 1.6 AST Transformation Example

2009-03-26T16:02:00.001-05:00

With Groovy 1.6 released and this article on InfoQ got me to try some new features.

AST Transformation
With Groovy 1.6 you can define local and global transformation using annotations. So let's define @AssertParamsNotNull annotation for method which would perform AST Transformation at compile time. It simply asserts parameters of method are not null. I am following groovy documentation here and blog entry here by Hamlet D'Arcy.

There are three components to defining local AST Transformation

Step 1: Define Annotation
Step 2: Define GroovyASTTransformation
Step 3: Test and Usage

Pre-requisites
Download Groovy 1.6
Download Groovy Eclipse plugin for 1.6, available from this update URL.

http://dist.groovy.codehaus.org/distributions/updateDev_1.6/

This url is different from their usual update url. I believe they will update their distributions update url (http://dist.codehaus.org/groovy/distributions/update/)

Step 1: Define Annotation @AssertParamsNotNull


package com.learn.groovy16.ast.local
import java.lang.annotation.Retention
import java.lang.annotation.RetentionPolicy
import java.lang.annotation.Target
import java.lang.annotation.ElementType
import org.codehaus.groovy.transform.GroovyASTTransformationClass
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(["com.learn.groovy16.ast.local.AssertParamsNotNullASTTransformation"])
public @interface AssertParamsNotNull{

One thing to notice is here defining the ASTTransformation class. It requires complete qualified class name. Rest of the things are standard annotation declaration.

While working on getting this running in my environment I faced following errors/issues.

Unknown Type: ANNOTATION_DEF at line ...
This error comes if for some reason eclipse environment is still using older groovy installation. Check your GROOVY_HOME points to Groovy 1.6. If you are able to run this through command line, but eclipse is giving your errors. Update your Groovy Eclipse plugin from dev update URL. You may have to wipe it clean for eclipse to understand it.

Expected '{' but was found...
This error comes if for some reason there funny newline characters. I think I got this error if right after "public @interface AssertParamsNotNull" if { is in the next line. I believe I got funny character in between those two. I changed the text file encoding to UTF-8 and the error went away. It allowed me to have braces on the next line.

Step 2: Define GroovyASTTransformation


package com.learn.groovy16.ast.local
import org.codehaus.groovy.transform.GroovyASTTransformation
import org.codehaus.groovy.ast.ASTNode
import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.transform.ASTTransformation
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.ast.MethodNode
import org.codehaus.groovy.ast.ClassNode
import org.codehaus.groovy.ast.Parameter
import org.codehaus.groovy.ast.stmt.Statement
import org.codehaus.groovy.ast.stmt.AssertStatement
import org.codehaus.groovy.ast.expr.BooleanExpression
import org.codehaus.groovy.ast.expr.NotExpression
import org.codehaus.groovy.ast.expr.VariableExpression
@GroovyASTTransformation(phase=CompilePhase.SEMANTIC_ANALYSIS)
public class AssertParamsNotNullASTTransformation implements ASTTransformation 
{
 public void visit(ASTNode[] nodes, SourceUnit source)
  {
   List methods = source.getAST()?.getMethods()
    methods.findAll{MethodNode method ->
     method.getAnnotations(new ClassNode(AssertParamsNotNull))
    }.each{MethodNode method -> 
      List existingStatements = method.getCode().getStatements()
      Parameter[] parameters = method.getParameters()
      parameters.eachWithIndex(){ parameter, i ->
       existingStatements.add(i, createAssertStatement(parameter))
      }
    }
  }
  
  public Statement createAssertStatement(Parameter parameter){
   return new AssertStatement(
    new BooleanExpression(
     new VariableExpression(parameter))
  }
}

This is where most of the trick is happening. This class implements ASTTransformation interface implementing visit(ASTNode[], SourceUnit source) method. This method gets all methods marked with AssertParamsNotNull annotation, iterates over all method, iterates over all method parameters and calls createAssertStatement(parameter). createAssertStatement(parameter) creates Statements equivalent to assert parameter and inserts it into the AST tree. All this happens at compile time. So notice line

@GroovyASTTransformation(phase=CompilePhase.SEMANTIC_ANALYSIS)

Creating Statement and Expression using API is really cumbersome. It is good they are working on builder support for the same.

Step 3: Test and Run


package com.learn.groovy16.ast.local  

@com.learn.groovy16.ast.local.AssertParamsNotNull
  
def foo(String var)
{
  println var
}
foo("Hello")
foo(null)

Note the annotation with fully qualified name.

Interesting thing is that all transformation happens at compile time. It does not stop

Other interesting AST Transformations...

@CatchExceptions(list=[exception1, exception2, ..],rethrow=AppException)
Define AST transformation to remove clutterred exception handling logic and rethrow ApplicationException as noted by parameters of annotation. I think the challenge here will be to build AST Statement and Expression.

@SecureAccess(Role=Manager)
Allows to define security over method execution.

@Trace(isTraceEnabled=$GlobalTraceParameter)
Emits out trace of method. Entered, parameter values, exit, return type. This only gets emitted if GlobalTraceParameter is set to Y

Grails, Flex and MyEclipse

2009-03-11T11:46:00.001-05:00

This is a small exercise that I undertook following documentation I found on grails website. Let's create small calculator application which adds two operands. The idea is to setup the flex plugin and see how it works.

I used following tools

MyEclipse with flex builder
Grails
Groovy Eclipse plugin
Grails Fex Plugin

Step 1: Install Groovy Eclipse plugin
MyEclipse 6.5 installation that I had did not came with groovy/grails plugin. One of the nice thing about MyEclipse is that it is eclipse with additional tools. Use the following plugin update URL to get groovy plugin.

Here is eclipse software update URL: http://dist.codehaus.org/groovy/distributions/update/.

This plugin is optional. You can use grails command line commands and that will work as well.

Step 2: Create Grails Project and Install Grails Flex Plugin
Next thing is to install Grails Flex plugin. This is still under development. The one I used is version 0.2.

Create Grails application using New --> Other... --> New Grails Project. This will create grails project.

Alternatively you an use grails command line to create app. Go to command line, change directory to project root folder in your workspace and type command:
              grails install-plugin flex.

This will install flex plugin for grails.

I was using MyEclipse with Flex Builder, it was easy to add flex nature to my project. To do that right click and add flex nature

Step 3: Create Calc UI
Now project is setup, we are ready to create calc.mxml. I created this under webapp folder so that it is visible.

Use following calc.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:RemoteObject id="ro" destination="calcService"/>
    <mx:Model id="myCalcData">
        <calc>
            <operand1>{number1.text}</operand1>
            <operand2>{number2.text}</operand2>
        </calc>
    </mx:Model>
    <mx:Panel>
        <mx:Form>
            <mx:FormItem label="Operand 1:">
                <mx:TextInput id="number1"/>
            </mx:FormItem>
            <mx:FormItem label="Operand 2:">
                <mx:TextInput id="number2"/>
            </mx:FormItem>
            <mx:Button label="Calculate" click="ro.calc(myCalcData.operand1,myCalcData.operand2)"/>
             <mx:Label text="Result: {ro.calc.lastResult} " fontWeight="bold"/>
        </mx:Form>
    </mx:Panel>
</mx:Application>

Step 4: Create Grails HelloWorldService
Create grails new service from IDE. You can also create service by typing command
grails create-service HelloService

public class CalcService
{
static expose = ['flex-remoting']
def calc(int number1, int number2) { return number1 + number2 }
}

Step 5: Update configuration
Next step is to update configuration file, Config.groovy to enables web tier compiler for flex. Again, I am using similar file as in the example provided on grails website.

// enables the webtier compiler for all environments

Step 6: Run using command line

We are almost done, type in command line at project root to run the projectgrails run-app

Type in browser: http://localhost:8080/MyGrails/calc.mxml

You may need to change port, application context if you are using different configuration.

Webservices with return type List

2008-05-25T12:05:00.000-05:00

I had written few web service which were returning List. Following are my findings when working with that.

Axis2 did not generate definition of T in wsdl. So to over come that problem I created a dummy method setT(T t) in order to get type definition of T generated in wsdl. Even though this is work around and it did not highlight or resolve the actual problem.

Axis2 generated List return type as anyType Object. Which created problem during client side code generation. The wsdl published the return type as anyType, Client had no way of determining what type of return object to be generated. So to be type safe I changed the implementation from List to T[]. Now while I believe type List will always be generated as anyType, I do think that List should not be generated as anyType. Tools should be smart enough to generate List as ArrayOfSpecificType in wsdl.
I think that is either due to two reasons,

Tools have not caught upto the JDK5
The specific way Java implements Generics, by using erasure, List becomes List which creates problem for the tool as tools have no way of identifying what type of list was that.

There are some other minor problems I faced while using array. I used method toArray(T[] t) to convert my List to Array. The method required me to pass a type T[] t. I created T[] t = new T[?]. I put number 3 - just randomly- to see if it all works. Service worked it generated response and it also appended two more elements named item1 and item2. I was clueless for a while where this sub elements are getting generated. Then it occurred to me the creation of array with three elements is putting it there. Finally, I settled with T[0]. But the problem with that is when result set was 0, client threw error stating unexpected element of type XXXXResponse. I believe this is a bug.

Struts: Creating Custom Validator

2007-07-02T11:36:00.000-05:00

Creating Custom Validator

Idea was to create custom password validator which will not allow certain words to be password. This included common words like password, companyname, abc, 123 etc. Also this requirement will be extended in future to disallow password being same as first name, last name, phone numbers, email address etc.

As direct implementation one can always override validate method in the Action Form class for the form. But since password field existed in multiple forms it made more sense to make generic validator which can be added to validation-rules.xml file for the application.

Add following section in validation-rules.xml file. This describes validation rule.

<validator name="custompassword" classname="com.web.helper.CustomValidationUtility" method="validatePassword" methodparams="java.lang.Object, org.apache.commons.validator.ValidatorAction, org.apache.commons.validator.Field, org.apache.struts.action.ActionMessages, org.apache.commons.validator.Validator, javax.servlet.http.HttpServletRequest" depends="" msg="errors.password"></validator>

Add the rule for the form that we need to validate.


<form name="/change-password">
...
<field property="newPassword" depends="custompassword">
...
</field>
</form>

Create custom validator classes as

package com.web.helper;

import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;

import javax.servlet.http.HttpServletRequest;

import org.apache.commons.validator.Field;
import org.apache.commons.validator.GenericValidator;
import org.apache.commons.validator.Validator;
import org.apache.commons.validator.ValidatorAction;
import org.apache.commons.validator.util.ValidatorUtils;
import org.apache.struts.action.ActionErrors;
import org.apache.struts.action.ActionMessages;
import org.apache.struts.validator.Resources;

public class CustomValidationUtility implements Serializable
{

public static boolean validatePassword(
Object bean,
ValidatorAction va,
Field field,
ActionMessages messages,
Validator validator,
HttpServletRequest request) {

System.out.println("Here now");
String value = ValidatorUtils.getValueAsString(bean, field.getProperty());
List invalidPasswordList = new ArrayList();
invalidPasswordList.add("password");
invalidPasswordList.add("companyname");
invalidPasswordList.add("abc");
invalidPasswordList.add("123");

if (!GenericValidator.isBlankOrNull(value)) {
try {
if (invalidPasswordList.contains(value)) {
messages.add(field.getKey(),
Resources.getActionError(
request,
va,
field));

return false;
}
} catch (Exception e) {
messages.add(field.getKey(),
Resources.getActionError(
request,
va,
field));
return false;
}
}

return true;
}
}

That's it.
Roadblocks:

One of the problem I faced while working on this in eclipse + wtp that the class was not included in class path automatically. I have to stop server + Compile Clean + Publish + Restart the server for each change I made. I struggled with following exceptions. First was loadValidationActionClass method threw ValidatorException. This is because the my ValidationAction class i.e. CustomValidationUtility was not in classpath. I hoped that WTP will take care of this dynamically, but for some reason it didn't

Second exception was regarding the parameter of my validation method, they have to exactly match as given in validation-rules.xml.

Third problem faced was whether to use ActionErrors or ActionMessages as parameter. There are numerous example out on the web for custom validator to use ActionErrors. It gave me NullPointerException. On debugging I found out that errors parameter in the method was null. I switched it to ActionMessages (which is parent class of ActionError) and it worked. (Of course, WTP made me do entire trip of stopping, cleaning, compiling, publishing, and starting)

Some other problems other people faced working on this (which I found out while google-ing): You have to be absolutely sure which version of Struts and commons validator you are using. Struts 1.1 goes with commons-validator 1.1 and Struts 1.2 goes with Commons-validator 1.3. People spent frustrating amount of effort to resolve error only to find out they have mismatched library.

Usage and/or Extension of above idea
If developing commons validator based validation mechanism, above methods can be used to come up with specific validation for those fields which are going to used at multiple places. Advantages of such mechanism is it will save you from writing and debugging java script, can come up with architecture where you can validate it with databases - though approach needs to be well thought out and used sparingly. (e.g. list of password could have been stored in database tables), comparison with other fields on the page. In such case, I believe that validate method in form should be put to use rather than generic validations. Also above mechanism for validation can be used when your validation is interdependent on multiple fields on the same form.

Problem with mapping Nested Objects using Struts

2006-07-18T14:56:00.000-05:00

I was working on designing a specific requirement where UI was a four step wizard. User will provide information in this four pages and submit the information to be stored in database. The way information was organized on UI (jsp) was different than the way it needed to be represented in DTO (or domain objects).

However, a good review pointed out that use of series of request.getParameter is not good idea. There must be good way to get the data map to Domain objects by using struts. If the data elements were linear ( not mapped and not indexed) Struts does map it quite well with ActionForm. You can use commons beanutils' BeanProperties.copyProperties(Object dest, Object source). But normally the object representation is not linear. Objects have association and multiplicity. (See simple example below)

It seems there is a problem with struts, when it comes to populating action form when jsp contains nested beans (or for that matter indexed beans). It simply does not work. Howerver, using nested tag library, i was able to get the JSP displayed properly with correct parameter name. The problem occurs when i try to submit the JSP page. It throws IllegalArgumentException: No Bean specified. I think I am not doing anything wrong. But, not sure.

Here is what I am trying to do. I have a JSP page with customer information(name, phone, email and address) where address contains(addres1, address2, city, state and zipcode). For purpose of simplicity I have kept all parameters as String. (Couldn't figure out easy way to link files, so I just pasted here.)

Address.java
package com.learning.struts.action;

public class Address {
private String address1;
private String address2;
private String city;
private String state;
private String zipcode;
public String getAddress1() {
return address1;
}
public void setAddress1(String address1) {
this.address1 = address1;
}
public String getAddress2() {
return address2;
}
public void setAddress2(String address2) {
this.address2 = address2;
}
public String getCity() {
return city;
}
public void setCity(String city) {
this.city = city;
}
public String getState() {
return state;
}
public void setState(String state) {
this.state = state;
}
public String getZipcode() {
return zipcode;
}
public void setZipcode(String zipcode) {
this.zipcode = zipcode;
}
}

CustomerDTO.java
package com.learning.struts.action;

public class CustomerDTO implements java.io.Serializable{
private String name;
private String phone;
private String email;
private Address address;
public Address getAddress() {
return address;
}
public void setAddress(Address address) {
this.address = address;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getPhone() {
return phone;
}
public void setPhone(String phone) {
this.phone = phone;
}
}

strust-config.xml <?xml version="1.0" encoding="ISO-8859-1" ?> <!DOCTYPE struts-config PUBLIC "-//Apache Software Foundation//DTD Struts Configuration 1.2//EN" "http://struts.apache.org/dtds/struts-config_1_2.dtd"> <struts-config> <form-beans> <form-bean name="customerForm" type="org.apache.struts.action.DynaActionForm"> <form-property name="page" type="java.lang.String"/> <form-property name="customer" type="com.learning.struts.action.CustomerDTO"/> </form-bean> <form-bean name="customerForm1" type="com.learning.struts.action.PageActionForm"> <form-property name="page" type="java.lang.String"/> <form-property name="customer" type="com.learning.struts.action.CustomerDTO"/> </form-bean> </form-beans> <global-forwards> </global-forwards> <action-mappings> <action path="/submit" type="com.learning.struts.action.IndexedNestedFormStrutsAction" validate="false" input="customer-address1.jsp" name="customerForm1"> <forward name="success" path="/jsp/index.jsp"/> </action> </action-mappings> </struts-config></blockquote> customer-address.jsp <%@ page language="java" contentType="text/html; charset=ISO-8859-1" pageEncoding="ISO-8859-1"%> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <%@ taglib uri="/WEB-INF/struts-html.tld" prefix="html" %> <%@ taglib uri="/WEB-INF/struts-nested.tld" prefix="nested" %> <html:html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <title>Insert title here</title> </head> <body> <html:form action="submit.do" method="post"> <html:hidden property="page" value="page-1"/> customer.name :<html:text property="customer.name" value="k"/> customer.phone: </p><html:text property="customer.phone" value="k"/> customer.email: </p><html:text property="customer.email" value="k"/> <nested:nest property="customer.address"> customer.address.address1: </p><nested:text property="address1" value="k" /> customer.address.address2: </p><nested:text property="address2" value="k"/> customer.address.city: </p><nested:text property="city" value="k" /> customer.address.state: </p><nested:text property="state" value="k" /> customer.address.zipcode: </p><nested:text property="zipcode" value="k" /> </nested:nest> <html:submit></html:submit> </html:form> </body> </html:html>

IndexedNestedFormStrutsAction.java

public class IndexedNestedFormStrutsAction extends Action {

public ActionForward execute(ActionMapping map, ActionForm form, HttpServletRequest req, HttpServletResponse res) throws Exception {
DynaActionForm daForm = (DynaActionForm)form;
CustomerDTO customer = (CustomerDTO)daForm.get("customer");
System.out.println("Customer : " + customer);
return map.findForward("success");
}
}

On debugging this, it throws up folloing exception :

java.lang.IllegalArgumentException: No bean specified
at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:751)
at org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:937)
at org.apache.commons.beanutils.BeanUtilsBean.populate(BeanUtilsBean.java:811)
at org.apache.commons.beanutils.BeanUtils.populate(BeanUtils.java:298)
at org.apache.struts.util.RequestUtils.populate(RequestUtils.java:493)
at org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:816)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:203)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)

But I thought that it may be problem with DynaActionForm so I created ActionForm derivative and try if it properly maps the nested bean. Here is the code for that:

PageActionFrom.java
package com.learning.struts.action;

import org.apache.struts.action.ActionForm;

public class PageActionForm extends ActionForm {
private String page;
private CustomerDTO customer;
public CustomerDTO getCustomer() {
return customer;
}
public void setCustomer(CustomerDTO customer) {
this.customer = customer;
}
public String getPage() {
return page;
}
public void setPage(String page) {
this.page = page;
}
}

But it does not work and though it generated the JSP correctly. When you do view source on generated html it will create proper dot notaion name for each input element. However, submitting and getting the values even inside ActionForm parameter of Action does not work.
I did find out some open source project in works which are working to deal with this problem. But they are in still 0.x version. I think this is something struts should do automatically rather than any plugin framework. Please correct me if I am wrong.

Random Thoughts: EJB3 Vs Spring

2006-07-11T19:16:00.000-05:00

Following are just my random thoughts. I am not trying to determine which one is better. I think it is pretty much decided that both are going co-exists.

Spring: IoC Container and Framework Library

I think it is good idea to understand Spring in two aspects.

First it is IoC container. This container is its core concepts. EJB as the SPEC is in direct competition. Question is: Do you want to use Spring IoC container or EJB Container?
Second aspects of the Spring Framework is all the helper classes which makes the development so easy. Like JdbcTemplate, Hibernate helper classes, etc. EJB does not provide such classes. Those gaps are filled up by the JBoss Seam Framework and the like. You can use EJB Container and still use Spring Framework Libraries.

So along with Spring Vs EJB, it is also Spring vs JBoss Seam and the like.

Spring Not a Spec
Spring, though opensource, is controlled by Interface 21, a commercial company. Interface 21 controls (atleast decides) what needs to be in next phase and hires programmer to do it. Don't get me wrong, I really like Spring. As a developer, it is really nice experience. How simple it makes to develop application and test it. But the fact remains that in the long run, we have to "wish" (or influence) that Interface 21 makes correct choices. Also remember that once we use spring we are using "tool specific classes and libraries" so we cannot decide to change it to different platform overnight.
Even though it uses its own library. It is based on very sound design principle. Design by Interface. Framework actually builds on top of it to provide re-usable code. For example, never worrying about handling connections.

With EJB, if you dont like a vendor you can still move to different vendor. If you dont like EJB at all depending upon your design you can still convert it to different technologies as there is not dependance on specific libraries. (Meaning dependency is there but on little higher level)

EJBs dont provide that kind of framework. It does not provide you reusable classes library. It does not provide helper classes for sending email for different implemenations.

Will Spring remain light?
But with all these introduction in Spring, how light is spring or how light will spring remain. But the answer lies in its core design. Since the core concepts behind it is Design by Interface and Inversion of Control. It allows to selectively use areas which we require. You dont want to use MVC dont incldue that jar. If you only want to use with regards to JDBC operations, only use spring-jdbc.jar. It will stay light as much your requirements demand it to remain light.

EJB: Market Availability
Though it is too early to say how light is JEE. It is just out and there are really few implementations available.

EJB still implements "Remote-ness"
One difference between these two choices still exists, that EJB3 will still do remoting. Spring on the other hand does not do remoting by default. So decision has to be made what kind of architecture is required keeping in mind business application scenario.

Thoughts about RPC
Also one thing to notice here is that RPC eventually will go out of fashion (IMHO). Ultimately, architectures like SOA where core concept is to exchange data rather than invoking behavior remotely. It is more loosely coupled, more reusable, and it also has advantage of platform independance.

Separating JPA out of EJB
One good things EJB Spec3 has done has separated JPA out of EJB Spec.
Again Hibernate will be JPA compliant in coming days. There are already articles floating around how to use Spring with JPA.

What are the other benefits of EJB3?
Container Managed Persistence
If you are using Container Managed Persistence, it manages the state of entities. Meaning it calls update insert methods on its own. It has container managed persistence. Meaning you dont have to call your insert update delete. Container will manage that for you. This is a really good advantage. With Spring Framework, you still need to call specific classes, SimpleJDBCTemplate.update, or session.update if you are using Hibernate. With EJB once configured with CMP, you dont do anything. You save that one method call. Calls will be made for you before and after business methods.

Message Driven Beans
With EJB Container you will have MessageDrivenBeans. i.e. you will have mechanisms to write consumers of Messages. With Spring you have to choose a separate provider for the same. Which gives you more flexibility. Spring delegates that work. Again principle on which spring is based: Do not re-write it if somebody is already doing it in good way. Rather than that just provide the loose interface and bunch of classes with re-usable code.

Clustering
Benefits of clustering and are anyways provided by application server. Even for that matter Tomcat provides clustering now. You can still develop Spring application and deploy it in different clusters in Tomcat. You dont need EJB Container to have component level clustering.

Spring is already a winner it. It has made EJB people change the spec. EJB3 now looks less similar to EJB2 and more similar to Spring. They have almost every aspect which spring got it correct. Aspects are now Interceptors. (Though this was any ways used by EJB container provider in their implementations, no programmatic mechanisms were provided for Bean Developer though)

XML vs Annotations
Transactions
With regards to Transaction declaration as Annotations. I must say, I like XML. No matter how much verbose. It is still good. Rather than my transaction strategy littered over code, i would like to have it one place/area.

Security
Dont let me wander in code to find out what security rules are implemented. Anyways, Spring has also come us with Annotations specific declarations allowing you to do quick and dirty work though.

Do we need IoC Container Spec?
Do we need a spec so other providers can come up with their own implementations of IoC container? Or have we already convoluted J2EE area with so many specifications and implementation?

Preference
Spring will work inside container and outside container. Spring is really great for small application because it is really easy to develop. With Spring 2.0's goal being easier XML Config files and allowing Annotations and its integration with AspectJ. Spring does have capability of using it in bits and pieces, meaning you may just use it in Middel Tier, you choose to use JMS features.

JEE spec will need a container. Also the choice depend upon the business case for which the application is getting developed. If the application is big in terms of scope or it is expected that it live long time while going multiple phase changes then I think it is better to stick with Standard Spec.

Thoughts: Open-Closed Principle

2006-06-24T01:13:00.000-05:00

Just read an article about Open Closed Principle[pdf]. It is good revision and insight into object oriented principles.

Very briefly, Open Closed Principle says to design your class which are close for modification and open for extension. As author suggests one way of doing this is to program to interfaces. So that the classes in design are closed for modification and open for extension.
How to ensure that the class designed is closed for modification? Though author points out that you can't close the class entirely. What is the generic questions checklist that we need to go through to verify that the class is closed for modification.

Is the class close for modification due to change in business requirements?
Business requirements change over time. The question we need to ask while reviewing the design is that have we designed the class in manner that it is able to display some flexibility of handling the changed requirement by extending it and not by modifying it. Are the safeguards in place which prevents changing the classes?

Will the changes in classes on which this class depends bring change in this class?
List down all classes which are used by the class under review. All the objects which are part of pre-condition for this class to operate.
Determine that how close is the current class' design in order to accomodate changes in these classes.
Key question: Are the contract between these classes defined by interfaces? (This leads to another principle of Dependancy Inversion Principle[pdf])

Will the changes in the classes which depends on this class bring change in this class?
List down all the classes which will use the current class.
Determine that how close is the current class' design in order to service the changing demands of these classes/objects.
Key question: Is Contract between these classes defined by interfaces?

Has design considered multiplicity requirements?
Lets say that sound contract exists between the classes on which the class depends and the classes which depends on this particular class.
Determine the impact of change in cardinality requirement of contract. A class may play different role. Does this class play role of "creator" of objects, "worker" on objects and/or "delegator" to objects. The contract defined in interface does it cover scenarios like: what if this class needs to "create" more than one objects in future. Is this class close enough for modification and open enough for extension to allow to "work" on multiple objects.

Again I think all this questions needs to be asked keeping in mind the domain for which it is getting designed. This questions are subjective to functional area application is trying to solve.

Configuring Subversion with Netbeans

2006-02-23T04:32:00.000-06:00

Configure Subversion with Netbeans

Step 1: Download Subversion profile from site http://vcsgeneric.netbeans.org/profiles/index.html
I configured it with Netbeans 4.1. The site also have profiles for other version of netbeans.

Step 2: Open Netbeans

Step 3: Click on Tools --> Update Center

Step 4: Select Install Manually downloaded Modules (.nbm Files)

Step 5: Click on Add and select the file that you downloaded. Finish the Wizard

Step 6: Click on Versioning --> Versioning Manager. In this popup add the working directory of the project and repository URL for Subversion. For instance I added c:\projects\projectname\src and repository as file:////svn ( mapped to c:\svn, if working on windows environment)

Step 7: Done. Now you can right click the project and view the files and status for the same.

Properties file or XML File

2006-02-21T04:37:00.000-06:00

While working on Java Projects we usually come to question whether to use properties file or to use XML files for reading configuration information.

Before we consider what solution should be used, first let's see what kind of the configuration data is involved.

Configuration data are range from simpler items like server urls, schema name, database environment, mailing servers, etc. This kind of data can be classified as name=value pair. No relationship or hierarchy exist between them.

The other cases of configuration data that might be used within project contains relationship within data elements. One of the example of such data is struts-config.xml. For this kind of data they will be more or less stored as Java Objects.

Once we made the classification decision is simple.

Use properties file for configuration data which is flat in nature. Data elements does not contain any relationship or hierarchy. Using XML file to store will involve parsing overhead without any significant advantage.

XML file usage is justified only when we are going to create Java Objects out of config data elements having relationship between them.

One of the pitfalls using XML file to store config file is to go on overkill and start integrating logic in XML tags. Make sure that it is only config data and you are not actually programming in XML.

Sun Open sourcing JES

2005-12-02T09:56:00.001-06:00

Sun has open sourced JES (Java Enterprise System) including Java application server platform and identity manager suite. CPCR7WHNFVTX