Thursday, May 09, 2013 | Posted by Kartik Shah at 10:18 PM | 0 comments
How to make Xcode use latest SVN Client
Wednesday, February 01, 2012 | Posted by Kartik Shah at 3:02 PM | 4 comments
JAXB - Unmarshal non root element
While generating JAX-WS web service client code using wsimport, it generates Object factory top level method input and output type. This restricts the use to marshal and unmarshal classes which are contained within the top level objects.
ObjectFactory.java
query.xml
MethodInput.java
JaxbUnmarshallerMethodInput.java
account.xml
Labels: java, jaxb, web services
Saturday, July 09, 2011 | Posted by Kartik Shah at 1:15 PM | 0 comments
SOA Composite deployment - Oracle SOA Suite
This post is going to be about very specific subject - SOA Composite deployment on Oracle SOA Suite. I am going to capture few issues faced while deploying SOA composite.
Sample Composite
For purpose of this blog, consider a simple SOA composite that does the following:
- Read entries from a database
- Create XML from the entries
- Store XML entries to another database
- Read XML entries
- Post each XML entry to a web service
There is good amount of workflow and business logic that makes it a perfect use case for doing it with BPEL, but that is beyond the purpose of this entry.
Composite Dependencies
Sample SOA composite had following dependencies:
- Couple of Datasource (one to read source data and other to store XML Entries)
- JMS Queue (Weblogic’s Uniform Distributed Queues)
Deployment Process
1. Datasources
Create Generic Datasources pointing to individual Oracle RAC node and a Multi Datasource using WLST scripts. (Most of the scripts are written for Weblogic 10.3.3, so haven’t explored newly introduced GridLinked Datasource in Weblogic 10.3.4.)
2. JMS Resources
Create JMS Server, JMS Module, Sub deployment, Connection Factory and JMS Queues using WLST scripts.
3. DBAdapter.rar
The composite uses DBAdapter connector to interact with databases, so add outbound connection pools the DBAdapter.rar pointing to actual Weblogic Datasources and redeploy application
4. Deploy Composite
Finally, deploy the composite using Enterprise Manager Console.
Lessons Learned
1. Server Start parameters
Using Enterprise Manager to deploy, the deployment window got stuck and never returned back. The issue, it turns out, is that the deployment was not able to communicate among instance servers. If SOA cluster is using multicast to communicate across weblogic instances, you will required following parameters on each server instance. The parameter adds well known address
-Dtangosol.coherence.wka1=soa01.mycompany.com -Dtangosol.coherence.wka2=soa02.mycompany.com -Dtangosol.coherence.localhost=soa01.mycompany.com -Xmx2048m
2. Failure updating DBAdapter.rar connector application
On attempt to create outbound connection pool pointing to weblogic datasource, activation fails complaining FileNotFoundException: Plan.xml. The file is present on the primary server instance, but it isn't replicated on other nodes. I am not sure why but for some reason the Plan.xml was required to present on all the nodes, even if deployment is being done from machine with AdminServer.
3. Failure on interaction with DB on some instances.
BINDING.JCA-12511
JCA Binding Component connection issue.
JCA Binding Component is unable to create an outbound JCA (CCI) connection.
apptools:InsertLogDB [ InsertLogDB_ptt::insert(TestLogCollection,TestLogCollection) ] : The JCA Binding Component was unable to establish an outbound JCA CCI connection due to the following issue: BINDING.JCA-12510 JCA Resource Adapter location error.
Unable to locate the JCA Resource Adapter via .jca binding file element <connection-factory/> The JCA Binding Component is unable to startup the Resource Adapter specified in the <connection-factory/> element: location='eis/DB/application-ds'. The reason for this is most likely that either 1) the Resource Adapters RAR file has not been deployed successfully to the WebLogic Application server or 2) the '<jndi-name>' element in weblogic-ra.xml has not been set to eis/DB/application-ds. In the last case you will have to add a new WebLogic JCA connection factory (deploy a RAR). Please correct this and then restart the Application ServerPlease make sure that the JCA connection factory and any dependent connection factories have been configured with a sufficient limit for max connections. Please also make sure that the physical connection to the backend EIS is available and the backend itself is accepting connections.
- Copy the Plan.xml containing all outbound connection pool to all nodes
- Update the connector application DBAdapter.rar
- Restart DBAdapter.rar and Application server instance
4. BAM Cluster Multicast misconfiguration
A misconfiguration on the Multicast address of BAM cluster resulted in Access Forbidden 403. The issue was that BAM server was trying to communicate with another environment cluster. (i.e. prod bam cluster trying to communicate with test bam cluster).
Here is the error:
<BEA-000141> <TCP/IP socket failure occurred while fetching statedump over HTTP from 142374575950937656S:10.50.XX.XXX:[XXXXX,XXXXX,-1,-1,-1,-1,-1]:soa:soa_server1.
java.io.FileNotFoundException: Response: '403: Forbidden' for url: ' http://10.50.XXX.XXX:16101/bea_wls_cluster_internal/psquare/p2?senderNum=3&lastSeqNum=0&PeerInfo=10,3,4&ServerName=soa_server1'
at weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:487)
at weblogic.cluster.HTTPExecuteRequest.connect(HTTPExecuteRequest.java:67)
at weblogic.cluster.HTTPExecuteRequest.run(HTTPExecuteRequest.java:83)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)at weblogic.work.ExecuteThread.run(ExecuteThread.java:176)
So make sure that the Mutlicast address and port combination used for BAM cluster and SOA cluster is unique. There is a Multicast test utility that you can use to test communication between two instance. More information about the utility is available here. Also refer this to troubleshoot Multicast configuration.
export CLASSPATH=${CLASSPATH}:[bea_home]/server/lib/weblogic.jar
On Machine A:java utils.MulticastTest -A [multicast address] -P [multicast port] -N TestServer1
On Machine B:java utils.MulticastTest -A [multicast address] -P [multicast port] -N TestServer2
Hope this helps!.
Wednesday, April 20, 2011 | Posted by Kartik Shah at 1:13 PM | 2 comments
OSB Project Structure
I am doing some development with Oracle Service Bus (OSB). The primary goal is to mediate web services using OSB. Oracle does provide set of eclipse plugins to develop OSB configuration - OEPE suite. One of question that I wanted answer was what will be a good project structure to organize various components of OSB configuration.
Typically, OEPE plugins gives two eclipse project templates:
- Oracle Service Bus Configuration Project
- Oracle Service Bus Project
OSB Config project is the top level project which can include various OSB Project. A minimal OSB project includes components like business service, proxy service and WSDL. It is important to come up with a good project structure since the component shows up as is on sbconsole. You probably don't want to see all the components at same level.
After this exercise, I ended up with following structure of the project that I wanted to share. So here it is,
The top level project - OSB Config Project - represents a business domain. So OSB configuration of all services serving a particular domain goes under that umbrella. Individual OSB Project contains mediation component for a given physical service. I used folders to group business and proxy services to allow for multiple proxy services for a given business service. Using this structure it becomes easy to manage configuration using sbconsole.
I ended up using the ant build script and import/export WLST script by tweaking the ones given as part of this book - The definitive guide to SOA. The build script can export configuration from local server or workspace. Import deploys it to the target server. The import/export script also takes care of variables that changes from environment to environment. OSB has a features to create customization XMLs. You can create customization XML per environment and run it post configuration package deployment.
Labels: oepe, osb, service bus, web services
Sunday, January 23, 2011 | Posted by Kartik Shah at 11:18 PM | 5 comments
Using Clojure to work with RSS feeds
As I mentioned in my last blog entry, I am trying to learn Clojure. I think it is an interesting challenge to internalize new thinking paradigm. So following entry/code is an attempt to learn the paradigms of the language. Drop a comment, if you have any suggestion.
I found that, while trying to learn new language, one of the challenge is to find interesting practice problem to solve. The practice problem which helps you understand the strength of the language. I think one such problem space Clojure's application would be interesting is parsing and data crunching.
Example Problem
So the problem I picked is a simple one. I heavily use google reader and for that matter most of my web reading happens as part subscribed RSS feeds. I find myself manually filtering the feed list to suite my interest at the time. This exercise creates parser for RSS feed. This parser creates Clojure records based data structure. Once you have all the feed represented in records structure you can write variety of functions to filter the feed or even apply intelligent web algorithms.
Organization of Code
I am looking for a good way to organize code in clojure. Just like Java packages, clojure has namespaces. But I have still not found a good way to organize clojure functions into files/namespaces. For now, I chose to create namespace per domain.
- rss.data.clj - Define records data structure and parser
- rss.feed.clj - Defines functions to get feeds
- rss.stat.clj - Defines functions to filter feeds (and may be some statistical algorithms)
(ns rss.data
(:require [clojure.xml :as xml])
(:import [java.io ByteArrayInputStream]))
(defrecord rss-channel [title description link lastbuilddate pubdate items])
(defrecord rss-item [title description link guid pubdate])
(defn create-rss-channel [map items]
(let [title (:title map)
pubDate (:pubDate map)
description (:description map)
link (:link map)
lastBuildDate (:lastBuildDate map)]
(rss-channel. title description link lastBuildDate pubDate items)))
(defn create-item [item-data]
(let [description (:description item-data)
pubDate (:pubDate item-data)
guid (:guid item-data)
link (:link item-data)
title (:title item-data)
]
(rss-item. title description link guid pubDate)))
(defn parse-item [item-data]
(create-item (zipmap (map :tag item-data) (map :content item-data))))
(defn parse-items [items-data]
(map parse-item items-data))
(defn parse-channel [channel-content]
(loop [x (:content channel-content)]
(create-rss-channel
(into {} (filter #(not (= :item (key %))) (zipmap (map :tag x) (map :content x))))
(parse-items (map :content (into [] (filter #(= :item (:tag %)) x)))))
))
(defn parse-rss [rss]
(let [rss-data (ByteArrayInputStream. (.getBytes rss "UTF-8"))]
(for [x (xml-seq (xml/parse rss-data))
:when (= :rss (:tag x))]
(map parse-channel (:content x)))))
feed.clj
(ns rss.feed
(:require [clj-http.client :as client]))
(defn get-feed [url]
(:body (client/get url)))
(defn get-feeds [urllist]
(pmap get-feed urllist))
stat.clj
(ns rss.stat)
(def keywords ["Clojure" "Groovy" "Python"])
(defn find-items [rss-item-list keyword]
(filter #(.contains (first (:title %)) keyword) (flatten rss-item-list)))
(defn find-items-keywords [rss-item-list keyword-list]
(flatten (map #(find-items rss-item-list %) keyword-list)))
Trying Delicious RSS
Let's use RSS feed published by delicious.com. All RSS feed can found here. Let's use the feed popular which got has tag "programming". (http://feeds.delicious.com/v2/rss/popular/programming)
This one only had few links. Let's try one which has lots of items. (http://feeds.delicious.com/v2/rss/tag/programming?count=100)
user=> (rss.stat/find-items-keywords (map :items (flatten (rss.data/parse-rss (rss.feed/get-feed "http://feeds.delicious.com/v2/rss/popular/programming")))) ["Clojure" "Python" "Groovy"])
(#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#thaiyoshi"], :pubdate ["Sun, 23 Jan 2011 16:53:58 +0000"]}
#:rss.data.rss-item{:title ["Writing Forwards Compatible Python Code | Armin Ronacher's Thoughts and Writings"], :description nil, :link ["http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/"], :guid ["http://www.delicious.com/url/c6fb2679bca6c193a41d92513f7edca1#papaeye"], :pubdate ["Sat, 22 Jan 2011 16:09:10 +0000"]})
Source Code available at GitHub (https://github.com/kartikshah/rss-feeder)
user=> (rss.stat/find-items-keywords (map :items (flatten (rss.data/parse-rss (rss.feed/get-feed "http://feeds.delicious.com/v2/rss/tag/programming?count=100")))) ["Clojure" "Python" "Groovy"])
(#:rss.data.rss-item{:title ["The Evolution of a Python Programmer : Aleks' Domain"], :description nil, :link ["http://metaleks.net/programming/the-evolution-of-a-python-programmer"], :guid ["http://www.delicious.com/url/15311aafa2207da0ffc770903091ccea#tsung"], :pubdate ["Mon, 24 Jan 2011 02:44:19 +0000"]}
#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#adharmad"], :pubdate ["Mon, 24 Jan 2011 02:43:07 +0000"]}
#:rss.data.rss-item{:title ["PySCeS: the Python Simulator for Cellular Systems"], :description nil, :link ["http://pysces.sourceforge.net/index.html"], :guid ["http://www.delicious.com/url/40e012b894d62cd8cb9ba6b95855b868#tiguco"], :pubdate ["Mon, 24 Jan 2011 02:39:24 +0000"]}
#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#lonstile"], :pubdate ["Mon, 24 Jan 2011 02:24:25 +0000"]}
Saturday, December 11, 2010 | Posted by Kartik Shah at 10:26 PM | 4 comments
How-To - IntelliJ Idea for Clojure/Compojure on Google App Engine
StrangeLoop conference really spiked my interest in learning Clojure. I wanted to find a way to do web development using Clojure. So this entry is primarily a how-to on setup of IntelliJ Idea to do Clojure using Compojure web framework, Leiningen build tool and Google App Engine deployment environment.
Step 1: Download Leiningen
Download Leiningen script file and follow instruction to put it in PATH.
Step 2: Install "La Clojure" and "Leiningen" plugin for IntelliJ Idea.
To do this go to Preferences -> Plugins. Go to "Available" Tab
Step 3: Setup Leiningen plugin properties
To setup Leiningen plugin properties in IntelliJ click on Preferences -> Leiningen and give the path to the executable.
Step 4: Create project template
Now with the installation complete, we are ready to create a new project. For now the project creation is done outside IntelliJ Idea using Lein on command line. This is the only step done outside the IntelliJ Idea. Choose the folder structure where your IntelliJ workspace is.
$ lein new gae-cmpjr
Created new project in: gae-cmpjr
You can see that Lein created project folder structure. In the project root it created a project.clj file used by Leiningen. Under src folder, it also created default namespace by the name of the project and created a core.clj file under it.
Step 5: Create project IntelliJ
Let's create IntelliJ Project, even though project structure exists pick the same folder path that you selected in step 4.
- File -> Project New... and select project from scratch
- Select both GAE and Clojure. Provide the path to the appengine sdk

Now basic setup for the project is done. You can browse the project files using IntelliJ. (Biggest benefit of La Clojure is it auto-inserts and matches brackets for very "brackety language" :-))
Step 6: Setup Leiningen to run from IntelliJ
To setup Leiningen from IntelliJ, click on Leiningen tool window. (Window -> Tool Windows -> Leiningen)
Step 7: Setup web development libraries
Setup the project to include Compojure and Ring libraries for development.
project.clj
(defproject gae-cmpjr "1.0.0-SNAPSHOT"
:description "Example for Compojure on GAE using IntelliJ"
:namespaces [gae-cmpjr.core]
:dependencies [[compojure "0.4.0-SNAPSHOT"]
[ring/ring-servlet "0.2.1"]]
:dev-dependencies [[leiningen/lein-swank "1.2.0-SNAPSHOT"]]
:compile-path "web/WEB-INF/classes"
:library-path "web/WEB-INF/lib")
core.clj
Step 8: GAE setup
(ns gae-cmpjr.core
(:gen-class :extends javax.servlet.http.HttpServlet)
(:use compojure.core
ring.util.servlet)
(:require [compojure.route :as route]))
(defroutes hello
(GET "/" [] "<h1>Hello World Wide Web!</h1>")
(route/not-found "Page not found"))
(defservice hello)
web.xml
<?xml version="1.0" encoding="UTF-8"?>appengine-web.xml
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">
<display-name>Compojure on GAE using IntelliJ</display-name>
<servlet>
<servlet-name>hello</servlet-name>
<servlet-class>gae_cmpjr.core</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>hello</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>
</web-app>
Step 9: Compile and Build war
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>compgae</application>
<version>v0-2</version>
</appengine-web-app>
From the Leiningen Tool window in Idea run following commands.
lein clean
lein deps
lein compile
Step 10: Run locally
To locally run click on already create "Run Configuration"
Step 11: To upload application to appengine
Tools -> Upload Appengine application

Saturday, September 25, 2010 | Posted by Kartik Shah at 11:11 PM | 1 comments
Revisiting JMX DSL with Groovy GEP3
In one of the previous blog post, I experimented with a sample DSL for JMX reporting using Jfree chart. Groovy 1.8.2 (Beta) has introduced command expression language. This GEP-3 essentially allows you to create command expression language which can further simplify language of DSL. The primary use of this is to aid in writing impromptu scripts to visually demo various MBean values. In this example we will see bar charts for module processing time for all web modules.
Command expression language allows you to alternate method name and parameter for even number of argument
e.g.
foo a1 a2 a3 a4 a5. Foo(a1).a2(a3).a4(a5)
You can find more example and explanation on GEP-3 page.
Revisiting JMX example so that I can simplify the language as
server.url "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" query "jboss.web:*" filter "j2eeType=WebModule"
def filteredModules = server.filteredModules
chart.type "Bar" modules filteredModules title "Module Processing Time" width 1200 height 700 refresh 500 attributes params labels graphLabels
chart.show()
Let's look at the supporting code.
import org.jfree.chart.ChartFactory
import groovy.swing.SwingBuilder
import org.jfree.data.category.DefaultCategoryDataset
import org.jfree.chart.plot.PlotOrientation as Orientation
import javax.swing.WindowConstants as WC
import javax.management.ObjectName
import javax.management.remote.JMXConnectorFactory
import javax.management.remote.JMXServiceURL as JmxUrl
import javax.naming.Context
class Chart {
def chartModules
def chartType
def chartAttributes = {m -> [m.processingTime, m.path]}
def chartLabels =["Time per Webapp", "Webapp", "Time"]
def chartOptions = [false, true, true]
def windowTitle
def w
def h
def refreshRate
def orientation = "VERTICAL"
def dataset
def modules(m) {
chartModules = m
this
}
def type(type) {
chartType = type
this
}
def attributes(attr) {
chartAttributes = attr
this
}
def labels (lbls) {
chartLabels = lbls
this
}
def options (opts) {
chartOptions = opts
this
}
def title (title) {
windowTitle = title
this
}
def width(width) {
w = width
this
}
def height(height) {
h = height
this
}
def refresh(r) {
refreshRate = r
this
}
void show(){
switch(chartType){
case "Bar": drawBarChart(); break;
default: break;
}
}
void drawBarChart(){
calculateData()
def chart = ChartFactory.createBarChart(*chartLabels, dataset, Orientation."${orientation}", *chartOptions)
def swing = new SwingBuilder()
def frame = swing.frame(title:windowTitle, defaultCloseOperation:WC.EXIT_ON_CLOSE){
panel(id:'canvas') {rigidArea(width:w, height:h)}
}
while(true){
calculateData()
chart.fireChartChanged()
frame.pack()
frame.show()
chart.draw(swing.canvas.graphics, swing.canvas.bounds)
sleep(refreshRate)
}
}
void calculateData(){
def newDataset = new DefaultCategoryDataset()
chartModules.each{ m ->
def dsCall = chartAttributes.call(m)
newDataset.addValue dsCall[0], 0, dsCall[1]
}
this.dataset = newDataset
}
}
class Server {
def server
def queryObjects
def moduleFilter
def filteredModules
def url(serverName){
server = JMXConnectorFactory.connect(new JmxUrl(serverName)).MBeanServerConnection
this
}
def query (queryString) {
queryObjects = new ObjectName(queryString)
this
}
def filter (filterString) {
String[] allNames = server.queryNames(queryObjects, null)
filteredModules = allNames.findAll { name -> name.contains(filterString)}.collect { new GroovyMBean(server, it)}
}
}
server = new Server()
chart = new Chart()
params = {m -> [m.processingTime, m.path]}
graphLabels = ["Time per Webapp", "Webapp", "Time"]
server.url "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" query "jboss.web:*" filter "j2eeType=WebModule"
def filteredModules = server.filteredModules
chart.type "Bar" modules filteredModules title "Module Processing Time" width 1200 height 700 refresh 500 attributes params labels graphLabels
chart.show()
Problems faced
For some reason with the current beta build I couldn't do arrays or closures inline. so I had to put them as separate variables. I do not think it is by design and as 1.8.X comes close to release these wrinkles will be worked out.

