Thursday, May 09, 2013 | | 0 comments

How to make Xcode use latest SVN Client

Recently working on an iOS project faced an issue importing project to subversion. All svn commands worked fine from command line, but would not work from Organizer. The issue turned out to be incompatibility between svn client version to the version on subversion server. Xcode was using svn client bundled within Xcode app, while I had later version of the client installed elsewhere available on path. 

Here is how you can point Xcode to use latest version of svn. 

Wednesday, February 01, 2012 | | 4 comments

JAXB - Unmarshal non root element

While generating JAX-WS web service client code using wsimport, it generates Object factory top level method input and output type. This restricts the use to marshal and unmarshal classes which are contained within the top level objects.

More often than not, top level Schema objects are non-domain specific objects like MethodInput/MethodOutput. You can unmarshal them using ObjectFactory

ObjectFactory.java

query.xml

MethodInput.java

JaxbUnmarshallerMethodInput.java

But if you want to unmarshal XML chunks of objects contained within those top level object, the same approach does not work, if those generated classes are not included as part of ObjectFactory or they do not have annotation @XmlRootElement on top of that.

account.xml

Option 1
So the obvious option is to add @XmlRootElement to any generated classes that you want to unmarshal directly, but when you are using wsdls are from an external source, idea of updating generated classes breaks the process. 

Option 2
Another option is to pass the child element's Node object to unmarshal 

 

Saturday, July 09, 2011 | | 0 comments

SOA Composite deployment - Oracle SOA Suite

This post is going to be about very specific subject - SOA Composite deployment on Oracle SOA Suite. I am going to capture few issues faced while deploying SOA composite. 


Sample Composite

For purpose of this blog, consider a simple SOA composite that does the following: 

  • Read entries from a database
  • Create XML from the entries
  • Store XML entries to another database
  • Read XML entries 
  • Post each XML entry to a web service 

There is good amount of workflow and business logic that makes it a perfect use case for doing it with BPEL, but that is beyond the purpose of this entry. 


Composite Dependencies

Sample SOA composite had following dependencies: 

  • Couple of Datasource (one to read source data and other to store XML Entries)
  • JMS Queue (Weblogic’s Uniform Distributed Queues) 

Deployment Process

1. Datasources

Create Generic Datasources pointing to individual Oracle RAC node and a Multi Datasource using WLST scripts. (Most of the scripts are written for Weblogic 10.3.3, so haven’t explored newly introduced GridLinked Datasource in Weblogic 10.3.4.)


2. JMS Resources

Create JMS Server, JMS Module, Sub deployment, Connection Factory and JMS Queues using WLST scripts.


3. DBAdapter.rar

The composite uses DBAdapter connector to interact with databases, so add outbound connection pools the DBAdapter.rar pointing to actual Weblogic Datasources and redeploy application


4. Deploy Composite

Finally, deploy the composite using Enterprise Manager Console. 


Lessons Learned 


1. Server Start parameters

Using Enterprise Manager to deploy, the deployment window got stuck and never returned back. The issue, it turns out, is that the deployment was not able to communicate among instance servers. If SOA cluster is using multicast to communicate across weblogic instances, you will required following parameters on each server instance. The parameter adds well known address

-Dtangosol.coherence.wka1=soa01.mycompany.com -Dtangosol.coherence.wka2=soa02.mycompany.com -Dtangosol.coherence.localhost=soa01.mycompany.com -Xmx2048m

2. Failure updating DBAdapter.rar connector application

On attempt to create outbound connection pool pointing to weblogic datasource, activation fails complaining FileNotFoundException: Plan.xml. The file is present on the primary server instance, but it isn't replicated on other nodes. I am not sure why but for some reason the Plan.xml was required to present on all the nodes, even if deployment is being done from machine with AdminServer. 


3. Failure on interaction with DB on some instances.  

Even after the deployment was successful, any DB activities from nodes other than primary node failed giving following error:

BINDING.JCA-12511
JCA Binding Component connection issue.
JCA Binding Component is unable to create an outbound JCA (CCI) connection.
apptools:InsertLogDB [ InsertLogDB_ptt::insert(TestLogCollection,TestLogCollection) ] : The JCA Binding Component was unable to establish an outbound JCA CCI connection due to the following issue: BINDING.JCA-12510 JCA Resource Adapter location error.
Unable to locate the JCA Resource Adapter via .jca binding file element <connection-factory/> The JCA Binding Component is unable to startup the Resource Adapter specified in the <connection-factory/> element:  location='eis/DB/application-ds'. The reason for this is most likely that either 1) the Resource Adapters RAR file has not been deployed successfully to the WebLogic Application server or 2) the '<jndi-name>' element in weblogic-ra.xml has not been set to eis/DB/application-ds. In the last case you will have to add a new WebLogic JCA connection factory (deploy a RAR). Please correct this and then restart the Application Server

Please make sure that the JCA connection factory and any dependent connection factories have been configured with a sufficient limit for max connections. Please also make sure that the physical connection to the backend EIS is available and the backend itself is accepting connections.

One noticeable thing on weblogic console was that under testing tab of DBAdapter.rar all outbound connection pools were not visible. The issue is that updated Plan.xml is not replicated to all other nodes. You have to manually copy the Plan.xml containing all outbound connection pool to all server instance nodes. 
Here is what you need to do:
  • Copy the Plan.xml containing all outbound connection pool to all nodes
  • Update the connector application DBAdapter.rar
  • Restart DBAdapter.rar and Application server instance


4. BAM Cluster Multicast misconfiguration

A misconfiguration on the Multicast address of BAM cluster resulted in Access Forbidden 403. The issue was that BAM server was trying to communicate with another environment cluster. (i.e. prod bam cluster trying to communicate with test bam cluster). 

Here is the error: 

<BEA-000141> <TCP/IP socket failure occurred while fetching statedump over HTTP from 142374575950937656S:10.50.XX.XXX:[XXXXX,XXXXX,-1,-1,-1,-1,-1]:soa:soa_server1.
java.io.FileNotFoundException: Response: '403: Forbidden' for url: '
 http://10.50.XXX.XXX:16101/bea_wls_cluster_internal/psquare/p2?senderNum=3&lastSeqNum=0&PeerInfo=10,3,4&ServerName=soa_server1'
        at weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:487)
        at weblogic.cluster.HTTPExecuteRequest.connect(HTTPExecuteRequest.java:67)
        at weblogic.cluster.HTTPExecuteRequest.run(HTTPExecuteRequest.java:83)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
      at weblogic.work.ExecuteThread.run(ExecuteThread.java:176)

So make sure that the Mutlicast address and port combination used for BAM cluster and SOA cluster is unique. There is a Multicast test utility that you can use to test communication between two instance. More information about the utility is available here. Also refer this to troubleshoot Multicast configuration.

export CLASSPATH=${CLASSPATH}:[bea_home]/server/lib/weblogic.jar
On  Machine A:

java utils.MulticastTest -A [multicast address] -P [multicast port] -N TestServer1
On Machine B:
java utils.MulticastTest -A [multicast address] -P [multicast port] -N TestServer2 

 

Hope this helps!. 


Wednesday, April 20, 2011 | | 2 comments

OSB Project Structure

I am doing some development with Oracle Service Bus (OSB). The primary goal is to mediate web services using OSB. Oracle does provide set of eclipse plugins to develop OSB configuration - OEPE suite. One of question that I wanted answer was what will be a good project structure to organize various components of OSB configuration.

Typically, OEPE plugins gives two eclipse project templates:

  • Oracle Service Bus Configuration Project
  • Oracle Service Bus Project

OSB Config project is the top level project which can include various OSB Project. A minimal OSB project includes components like business service, proxy service and WSDL. It is important to come up with a good project structure since the component shows up as is on sbconsole. You probably don't want to see all the components at same level.

After this exercise, I ended up with following structure of the project that I wanted to share. So here it is,

The top level project - OSB Config Project - represents a business domain. So OSB configuration of all services serving a particular domain goes under that umbrella. Individual OSB Project contains mediation component for a given physical service. I used folders to group business and proxy services to allow for multiple proxy services for a given business service. Using this structure it becomes easy to manage configuration using sbconsole.

I ended up using the ant build script and import/export WLST script by tweaking the ones given as part of this book - The definitive guide to SOA. The build script can export configuration from local server or workspace. Import deploys it to the target server. The import/export script also takes care of variables that changes from environment to environment. OSB has a features to create customization XMLs. You can create customization XML per environment and run it post configuration package deployment.

Sunday, January 23, 2011 | | 5 comments

Using Clojure to work with RSS feeds

As I mentioned in my last blog entry, I am trying to learn Clojure. I think it is an interesting challenge to internalize new thinking paradigm. So following entry/code is an attempt to learn the paradigms of the language. Drop a comment, if you have any suggestion.

I found that, while trying to learn new language, one of the challenge is to find interesting practice problem to solve. The practice problem which helps you understand the strength of the language. I think one such problem space Clojure's application would be interesting is parsing and data crunching.

Example Problem
So the problem I picked is a simple one. I heavily use google reader and for that matter most of my web reading happens as part subscribed RSS feeds. I find myself manually filtering the feed list to suite my interest at the time. This exercise creates parser for RSS feed. This parser creates Clojure records based data structure. Once you have all the feed represented in records structure you can write variety of functions to filter the feed or even apply intelligent web algorithms.

Organization of Code
I am looking for a good way to organize code in clojure. Just like Java packages, clojure has namespaces. But I have still not found a good way to organize clojure functions into files/namespaces. For now, I chose to create namespace per domain.

  • rss.data.clj - Define records data structure and parser
  • rss.feed.clj - Defines functions to get feeds
  • rss.stat.clj - Defines functions to filter feeds (and may be some statistical algorithms)
data.clj

(ns rss.data
(:require [clojure.xml :as xml])
(:import [java.io ByteArrayInputStream]))

(defrecord rss-channel [title description link lastbuilddate pubdate items])
(defrecord rss-item [title description link guid pubdate])

(defn create-rss-channel [map items]
(let [title (:title map)
pubDate (:pubDate map)
description (:description map)
link (:link map)
lastBuildDate (:lastBuildDate map)]
(rss-channel. title description link lastBuildDate pubDate items)))

(defn create-item [item-data]
(let [description (:description item-data)
pubDate (:pubDate item-data)
guid (:guid item-data)
link (:link item-data)
title (:title item-data)
]
(rss-item. title description link guid pubDate)))

(defn parse-item [item-data]
(create-item (zipmap (map :tag item-data) (map :content item-data))))

(defn parse-items [items-data]
(map parse-item items-data))

(defn parse-channel [channel-content]
(loop [x (:content channel-content)]
(create-rss-channel
(into {} (filter #(not (= :item (key %))) (zipmap (map :tag x) (map :content x))))
(parse-items (map :content (into [] (filter #(= :item (:tag %)) x)))))
))

(defn parse-rss [rss]
(let [rss-data (ByteArrayInputStream. (.getBytes rss "UTF-8"))]
(for [x (xml-seq (xml/parse rss-data))
:when (= :rss (:tag x))]
(map parse-channel (:content x)))))

feed.clj

(ns rss.feed
(:require [clj-http.client :as client]))

(defn get-feed [url]
(:body (client/get url)))

(defn get-feeds [urllist]
(pmap get-feed urllist))

stat.clj

(ns rss.stat)

(def keywords ["Clojure" "Groovy" "Python"])

(defn find-items [rss-item-list keyword]
(filter #(.contains (first (:title %)) keyword) (flatten rss-item-list)))

(defn find-items-keywords [rss-item-list keyword-list]
(flatten (map #(find-items rss-item-list %) keyword-list)))

Trying Delicious RSS
Let's use RSS feed published by delicious.com. All RSS feed can found here. Let's use the feed popular which got has tag "programming". (http://feeds.delicious.com/v2/rss/popular/programming)

user=> (rss.stat/find-items-keywords (map :items (flatten (rss.data/parse-rss (rss.feed/get-feed "http://feeds.delicious.com/v2/rss/popular/programming")))) ["Clojure" "Python" "Groovy"])
(#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#thaiyoshi"], :pubdate ["Sun, 23 Jan 2011 16:53:58 +0000"]}
#:rss.data.rss-item{:title ["Writing Forwards Compatible Python Code | Armin Ronacher's Thoughts and Writings"], :description nil, :link ["http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/"], :guid ["http://www.delicious.com/url/c6fb2679bca6c193a41d92513f7edca1#papaeye"], :pubdate ["Sat, 22 Jan 2011 16:09:10 +0000"]})
This one only had few links. Let's try one which has lots of items. (http://feeds.delicious.com/v2/rss/tag/programming?count=100)

user=> (rss.stat/find-items-keywords (map :items (flatten (rss.data/parse-rss (rss.feed/get-feed "http://feeds.delicious.com/v2/rss/tag/programming?count=100")))) ["Clojure" "Python" "Groovy"])

(#:rss.data.rss-item{:title ["The Evolution of a Python Programmer : Aleks' Domain"], :description nil, :link ["http://metaleks.net/programming/the-evolution-of-a-python-programmer"], :guid ["http://www.delicious.com/url/15311aafa2207da0ffc770903091ccea#tsung"], :pubdate ["Mon, 24 Jan 2011 02:44:19 +0000"]}
#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#adharmad"], :pubdate ["Mon, 24 Jan 2011 02:43:07 +0000"]}
#:rss.data.rss-item{:title ["PySCeS: the Python Simulator for Cellular Systems"], :description nil, :link ["http://pysces.sourceforge.net/index.html"], :guid ["http://www.delicious.com/url/40e012b894d62cd8cb9ba6b95855b868#tiguco"], :pubdate ["Mon, 24 Jan 2011 02:39:24 +0000"]}
#:rss.data.rss-item{:title ["Hidden features of Python - Stack Overflow"], :description nil, :link ["http://stackoverflow.com/questions/101268/hidden-features-of-python/102062"], :guid ["http://www.delicious.com/url/16bbee33bbf4f6a03448058c5fd9f461#lonstile"], :pubdate ["Mon, 24 Jan 2011 02:24:25 +0000"]}
Source Code available at GitHub (https://github.com/kartikshah/rss-feeder)
Blogged with the Flock Browser

Saturday, December 11, 2010 | | 4 comments

How-To - IntelliJ Idea for Clojure/Compojure on Google App Engine

StrangeLoop conference really spiked my interest in learning Clojure. I wanted to find a way to do web development using Clojure. So this entry is primarily a how-to on setup of IntelliJ Idea to do Clojure using Compojure web framework, Leiningen build tool and Google App Engine deployment environment.

Step 1: Download Leiningen
Download Leiningen script file and follow instruction to put it in PATH.

Step 2: Install "La Clojure" and "Leiningen" plugin for IntelliJ Idea.
To do this go to Preferences -> Plugins. Go to "Available" Tab
compojure-gae-1.png

Step 3: Setup Leiningen plugin properties
To setup Leiningen plugin properties in IntelliJ click on Preferences -> Leiningen and give the path to the executable.

Step 4: Create project template
Now with the installation complete, we are ready to create a new project. For now the project creation is done outside IntelliJ Idea using Lein on command line. This is the only step done outside the IntelliJ Idea. Choose the folder structure where your IntelliJ workspace is.


     $ lein new gae-cmpjr
     Created new project in: gae-cmpjr

You can see that Lein created project folder structure. In the project root it created a project.clj file used by Leiningen. Under src folder, it also created default namespace by the name of the project and created a core.clj file under it.

Step 5: Create project IntelliJ
Let's create IntelliJ Project, even though project structure exists pick the same folder path that you selected in step 4.
  • File -> Project New... and select project from scratch
  • Select both GAE and Clojure. Provide the path to the appengine sdk
compojure-gae-2.png

Now basic setup for the project is done. You can browse the project files using IntelliJ. (Biggest benefit of La Clojure is it auto-inserts and matches brackets for very "brackety language" :-))

Step 6: Setup Leiningen to run from IntelliJ
To setup Leiningen from IntelliJ, click on Leiningen tool window. (Window -> Tool Windows -> Leiningen)

Step 7: Setup web development libraries
Setup the project to include Compojure and Ring libraries for development.
project.clj

(defproject gae-cmpjr "1.0.0-SNAPSHOT"
  :description "Example for Compojure on GAE using IntelliJ"
  :namespaces [gae-cmpjr.core]
  :dependencies [[compojure "0.4.0-SNAPSHOT"]
                 [ring/ring-servlet "0.2.1"]]
  :dev-dependencies [[leiningen/lein-swank "1.2.0-SNAPSHOT"]]
  :compile-path "web/WEB-INF/classes"
  :library-path "web/WEB-INF/lib")

core.clj

(ns gae-cmpjr.core
  (:gen-class :extends javax.servlet.http.HttpServlet)
  (:use compojure.core
        ring.util.servlet)
  (:require [compojure.route :as route]))

(defroutes hello
  (GET "/" [] "<h1>Hello World Wide Web!</h1>")
  (route/not-found "Page not found"))

(defservice hello)
Step 8:  GAE setup

web.xml
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
            http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
           version="2.5">
    <display-name>Compojure on GAE using IntelliJ</display-name>

    <servlet>
        <servlet-name>hello</servlet-name>
        <servlet-class>gae_cmpjr.core</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>hello</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>
</web-app>
appengine-web.xml

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  <application>compgae</application>
  <version>v0-2</version>
</appengine-web-app>
Step 9: Compile and Build war

From the Leiningen Tool window in Idea run following commands.

lein clean
lein deps
lein compile


Step 10: Run locally
To locally run click on already create "Run Configuration"

Step 11: To upload application to appengine
Tools -> Upload Appengine application
compojure-gae-4.png
Blogged with the Flock Browser

Saturday, September 25, 2010 | | 1 comments

Revisiting JMX DSL with Groovy GEP3

In one of the previous blog post, I experimented with a sample DSL for JMX reporting using Jfree chart. Groovy 1.8.2 (Beta) has introduced command expression language. This GEP-3 essentially allows you to create command expression language which can further simplify language of DSL. The primary use of this is to aid in writing impromptu scripts to visually demo various MBean values. In this example we will see bar charts for module processing time for all web modules.

Command expression language allows you to alternate method name and parameter for even number of argument
e.g.
foo a1 a2 a3 a4 a5. Foo(a1).a2(a3).a4(a5)
You can find more example and explanation on GEP-3 page.

Revisiting JMX example so that I can simplify the language as


server.url "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" query "jboss.web:*" filter "j2eeType=WebModule"
def filteredModules = server.filteredModules

chart.type "Bar" modules filteredModules title "Module Processing Time" width 1200 height 700 refresh 500 attributes params labels graphLabels
chart.show()

Let's look at the supporting code.

import org.jfree.chart.ChartFactory
import groovy.swing.SwingBuilder

import org.jfree.data.category.DefaultCategoryDataset
import org.jfree.chart.plot.PlotOrientation as Orientation
import javax.swing.WindowConstants as WC
import javax.management.ObjectName
import javax.management.remote.JMXConnectorFactory
import javax.management.remote.JMXServiceURL as JmxUrl
import javax.naming.Context

class Chart {
    def chartModules
      def chartType
    def chartAttributes = {m -> [m.processingTime, m.path]}
    def chartLabels =["Time per Webapp", "Webapp", "Time"]
    def chartOptions = [false, true, true]
    def windowTitle
    def w
    def h
    def refreshRate
    def orientation = "VERTICAL"
    def dataset

    def modules(m) {
          chartModules = m
          this
      }

      def type(type) {
        chartType = type
         this
      }

      def attributes(attr) {
        chartAttributes = attr
        this
      }

      def labels (lbls) {
        chartLabels = lbls
        this
      }

      def options (opts) {
        chartOptions = opts
        this
      }

      def title (title) {
        windowTitle = title
        this
      }

      def width(width) {
        w = width
        this
      }

      def height(height) {
        h = height
        this
      }

      def refresh(r) {
        refreshRate = r
        this
      }

    void show(){
        switch(chartType){
            case "Bar": drawBarChart(); break;
            default: break;
        }
    }

    void drawBarChart(){
        calculateData()
        def chart = ChartFactory.createBarChart(*chartLabels, dataset, Orientation."${orientation}", *chartOptions)
        def swing = new SwingBuilder()
        def frame = swing.frame(title:windowTitle, defaultCloseOperation:WC.EXIT_ON_CLOSE){
            panel(id:'canvas') {rigidArea(width:w, height:h)}
        }
        while(true){
            calculateData()
            chart.fireChartChanged()
            frame.pack()
            frame.show()
            chart.draw(swing.canvas.graphics, swing.canvas.bounds)
            sleep(refreshRate)
        }
    }
    void calculateData(){
        def newDataset = new DefaultCategoryDataset()
        chartModules.each{ m ->
            def dsCall = chartAttributes.call(m)
            newDataset.addValue dsCall[0], 0, dsCall[1]
        }
        this.dataset = newDataset
    }
}


class Server {
    def server
    def queryObjects
    def moduleFilter
    def filteredModules

    def url(serverName){
    server = JMXConnectorFactory.connect(new JmxUrl(serverName)).MBeanServerConnection
    this
    }

    def query (queryString) {
        queryObjects = new ObjectName(queryString)
       this
    }

    def filter (filterString) {
         String[] allNames = server.queryNames(queryObjects, null)
        filteredModules = allNames.findAll { name -> name.contains(filterString)}.collect { new GroovyMBean(server, it)}
    }
}

server = new Server()
chart = new Chart()
params = {m -> [m.processingTime, m.path]}
graphLabels = ["Time per Webapp", "Webapp", "Time"]

server.url "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" query "jboss.web:*" filter "j2eeType=WebModule"
def filteredModules = server.filteredModules

chart.type "Bar" modules filteredModules title "Module Processing Time" width 1200 height 700 refresh 500 attributes params labels graphLabels
chart.show()

Problems faced
For some reason with the current beta build I couldn't do arrays or closures inline. so I had to put them as separate variables. I do not think it is by design and as 1.8.X comes close to release these wrinkles will be worked out.
Blogged with the Flock Browser