Revisit Jmx Groovy DSL - Using AstTransformation

In the previous blog entry, we created Jmx Charting DSL. Let visit some aspects of creating DSL using groovy.

In the previous blog example we created DSL using ExpandoMetaClass. Users will define the script starting with node jmx { }. With the use of ExpandoMetaClass we added the dynamic method which in turn delegated to class JmxClosureDelegate. Here is the snippet of the code.

static void runEngine(File dsl){
    Script dslScript = new GroovyShell().parse(dsl.text)
    dslScript.metaClass = createExpandoMetaClass(dslScript.class, {
      ExpandoMetaClass emc ->
        emc.jmx = {
          Closure cl ->
            cl.delegate = new JmxClosureDelegate()
            cl.resolveStrategy = Closure.DELEGATE_FIRST
            cl()
        }
    })
    dslScript.run()
   }

What we essentially did here:

• Using GroovyShell it parses the script file passed as input.
• Defines the ExpandoMetaClass and adds method by name "jmx" having closure as parameter.

So the script file you wrote gets a dynamic method injected into it using ExpandoMetaClass. This happens at runtime.

Getting rid of the Commas
The problem with creating methods with two arguments is that you have to specify commas between them. For example,

     server "nameofserver", {
          ...
     } 

Undoubtedly commas clutter the langauge grammer. To get rid of the commas we used trick provided on Groovy users list.

//To avoid using "," between String and Closure argument  
  def methodMissing(String name, args) {  
    return [name, args[0]]  
  }  

Using AstTransformation
Now lets explore a different possibility. You can achieve similar result using AstTransformation at compile time. The goal remains the same and that is to add method with name "jmx" with closure parameter.

First we will define the annotation.

//import statements skipped for brevity
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(["info.kartikshah.jmx.ast.JmxDslTransformation"])
public @interface UseJmxDsl {}

Transformation Class

This transformation class needs to perform two activities:

• Add method jmx(Closure cl) method
• Invoke the script method being defined

We need to generate AST statements for following snippet of code.

     jmx = {  
          Closure cl ->  
            cl.delegate = new JmxClosureDelegate()  
            cl.resolveStrategy = Closure.DELEGATE_FIRST  
            cl()  
        }  

We will use AstBuilder's buildFromSpec option to generate it. (AstBuilders added with Groovy 1.7 definitely makes generating statement structure relatively easy and clutter free. Not to mention it is also an example of DSL added to the groovy language :-) )

@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)
class JmxDslTransformation implements ASTTransformation {
  static int PUBLIC = 1
  static int STATIC = 8

  void visit(ASTNode[] astNodes, SourceUnit sourceUnit) {
    //Add method jmx(Closure cl)
    ClassNode declaringClass = astNodes[1].declaringClass
    MethodNode jmxMethod = makeMethod()
    declaringClass.addMethod(jmxMethod)

    //Insert method call inside run method of the script class
    MethodNode annotatedMethod = astNodes[1]
    List<MethodNode> allMethods = sourceUnit.AST?.classes*.methods.flatten()
    MethodNode runMethod = allMethods.find{ MethodNode method ->
      method.name == "run"
    }
    List existingStatements = runMethod.getCode().getStatements()
    existingStatements.add(0, createMethodCall(annotatedMethod))
  }

  Statement createMethodCall(MethodNode methodNode){
    def statementAst = new AstBuilder().buildFromSpec {
      expression{
         methodCall {
           variable "this"
           constant methodNode.name
           argumentList {}
         }
      }
    }
    Statement stmt = statementAst[0]
    stmt
  }

  MethodNode makeMethod() {
    def ast = new AstBuilder().buildFromSpec {
      method('jmx', PUBLIC | STATIC, Void.TYPE) {
        parameters {
          parameter 'cl': Closure.class
        }
        exceptions {}
        block {
          expression {
            binary {
              property {
                variable "cl"
                constant "delegate"
              }
              token "="
              constructorCall(JmxClosureDelegate.class) {
                argumentList()
              }
            }
          }
          expression {
            binary {
              property {
                variable "cl"
                constant "resolveStrategy"
              }
              token "="
              property {
                classExpression Closure
                constant "DELEGATE_FIRST"
              }
            }
          }
          expression {
            methodCall {
              variable "cl"
              constant "call"
              argumentList {}
            }
          }
        }
      }
    }
    MethodNode jmxMethod = ast[0]
    jmxMethod
  }
}

Why use AstTransformation?
The question is why one would want to use AstTransformation when you can add method during runtime. For the given scenario, it is correct that you want to stick with adding method runtime. But consider scenario where you want to "redefine" meaning of Groovy's syntax. For example like following imaginary script using Statement Labels to add more readability to your DSL syntax.

@info.kartikshah.jmx.ast.UseJmxDsl
runDsl () {
  jmx {
    setup:
      server "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector"
      query "jboss.web:*"
      findAll "j2eeType=Servlet"

    draw:
    chart {
        chartType="Bar"
        attributes={m-> [m.loadTime, m.objectName.find("name=([^,]*)"){it[1]}]}
        labels=["Load Time per Servlet", "Servlet", "Time"]
        options=[false, true, true]
        windowTitle="JBoss Servlet Processing Time"
        width=1200
        height=700
        orientation="HORIZONTAL"
        refreshRate=5000
        show()
      }
  }
}

Spock Framework does similar twist by redefining meaning of existing construct.

With this type of language structure you will end up defining your own set of keywords, supporting parser and few AstTransformation to change the meaning of existing Groovy Syntax.

Blogged with the Flock Browser

A Groovy DSL - JMX Reporting

In previous blog example, using JFreeChart, JMX and SwingBuilder we came up with dashboard type utility. With this post we will explore how to write small DSL which will allow to report on JMX using reporting charts. The DSL will using Groovy's SwingBuilder to draw JFree Chart to report on various MBeans. This type DSL can be used to write small scripts to monitor behavioral aspect of application server (or any JMX based application).

Simple DSL
First we will chart out how we want our domain language to look like. DSL syntax can be done multiple ways. It is necessary that you play with the syntax to come up with one that works for the scenario. For our example, here is the first cut that we will use.

Essentially, the script reports processing time of all web modules defined on (in this example - JBoss) application server using Bar Chart.

jmx {
    server "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" {
      query "jboss.web:*" {
        findAll "j2eeType=WebModule" {
          chart{
            chartType="Bar"
            attributes={m-> [m.processingTime, m.path]}
            labels=["Time per Webapp", "Webapp", "Time"]
            options=[false, true, true]
            windowTitle="JBoss Module Processing Time"
            width=1200
            height=700
            refreshRate=5000
            show()
          }
        }
      }
    }
  }

Other Use cases
Target users for this DSL are system administrators who can write simple scripts to monitor and/or report on various aspects of app server instances graphically.

Some other use cases for this type of DSL

• Compare Processing Time of Web Application
• Compare Load Time of Servlets
• Compare Response Time
• Compare memory usage
  
• Compare total requests

DSL Engine
With Groovy there are multiple ways to write domain specific language as described in groovy documentation here

We will use nested closure approach. More information about nested closure here.

Here we use ExpandoMetaClass and series of Delegate classes to handle each closure. In the sample script above read each node (eg. jmx, server, query) as a dynamic method call with one or two arguments. For example jmx has a string argument followed by closure argument. String argument is we just store in instance variable. For closure argument we delegate the handling to separate Delegate Class. We follow the same pattern for the rest of the nested nodes.

First up Engine
This is the main class for the DSL. It gets the file passed as command args and pass it to GroovyShell to create Script object. It uses ExpandoMetaClass to dynamically add methods/closure. It add adds jmx closure and sets properties for handling closure.

class JmxReportingDslEngine {
  
  static main(String[] args){
    if(args.length != 1)
    {
      println("Usage: JmxReportingDslEngine <ScriptFileName>")
    }
    runEngine(new File(args[0]))
  }
  
  static void runEngine(File dsl){
    Script dslScript = new GroovyShell().parse(dsl.text)
    dslScript.metaClass = createExpandoMetaClass(dslScript.class, {
      ExpandoMetaClass emc ->
        emc.jmx = {
          Closure cl ->
            cl.delegate = new JmxClosureDelegate()
            cl.resolveStrategy = Closure.DELEGATE_FIRST
            cl()
        }
    })
    dslScript.run()
   }

  static ExpandoMetaClass createExpandoMetaClass(Class clazz, Closure cl){
    ExpandoMetaClass emc = new ExpandoMetaClass(clazz, false)
    cl(emc)
    emc.initialize()
    return emc
  }
}

Delegates

Next up we write series of Delegate Class responsible for handling each node of the language.

JMXClosureDelegate
This handles closure passed to jmxtag. It instantiates MBeanServerConnection and passes the reference down to the delegate chain.

class JmxClosureDelegate {
  //To avoid using "," between String and Closure argument
  def methodMissing(String name, args) {
    return [name, args[0]]
  }
  void server(param){
    def (serverUrl, cl) = param
    def server = JMXConnectorFactory.connect(new JmxUrl(serverUrl),env).MBeanServerConnection
    cl.delegate = new JmxServerClosureDelegate(server)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }

JMXServerClosureDelegate
This delegate handles closure passed to server tag.

class JmxServerClosureDelegate {
  def server
  JmxServerClosureDelegate(server){
    this.server = server
  }
  def methodMissing(String name, args) {
   return [name, args[0]]
  }
  void query(param){
    def (objectName, cl) = param
    def query = new ObjectName(objectName)
    String[] allNames = server.queryNames(query, null)
    cl.delegate = new JmxQueryClosureDelegate(allNames, server)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }
}

JmxQueryClosureDelegate
and so on...

class JmxQueryClosureDelegate {
  def allNames
  def server
  JmxQueryClosureDelegate(allNames, server){
    this.allNames = allNames
    this.server = server
  }
  def methodMissing(String name, args) {
    return [name, args[0]]
  }
  void findAll(param){
    def (filter, cl) = param
    def modules = allNames.findAll{ name ->
          name.contains(filter)
      }.collect{ new GroovyMBean(server, it) }
    cl.delegate = new JmxFindAllClosureDelegate(modules)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }
}

JmxFindAllClosureDelegate

class JmxFindAllClosureDelegate {
  def modules
  JmxFindAllClosureDelegate(modules){
    this.modules = modules
  }
  void chart(Closure cl){
    cl.delegate = new ChartDelegate(modules)
    cl.resolveStrategy = Closure.DELEGATE_FIRST
    cl()
  }
}

ChartDelegate
This class

• Creates the dataset from the MBean values
• Creates the chart with the dataset values
• Creates the external frame using SwingBuilder

class ChartDelegate {

  def modules
  def chartType
  def attributes
  def labels
  def options
  def windowTitle
  def width
  def height
  def refreshRate
  def orientation = "VERTICAL"
  def dataset
  ChartDelegate(modules){
    this.modules = modules
  }
  void show(){
    switch(chartType){
      case "Bar": drawBarChart(); break;
     //TODO:Add more chart types
      default: break;
    }
  }
  void drawBarChart(){
    calculateData()
    def chart = ChartFactory.createBarChart(*labels, dataset, Orientation."${orientation}", *options)
    def swing = new SwingBuilder()
    def frame = swing.frame(title:windowTitle, defaultCloseOperation:WC.EXIT_ON_CLOSE){
      panel(id:'canvas') {rigidArea(width:width, height:height)}
    }
    while(true){
      calculateData()
      chart.fireChartChanged()
      frame.pack()
      frame.show()
      chart.draw(swing.canvas.graphics, swing.canvas.bounds)
      sleep(refreshRate)
    }
  }
  void calculateData(){
    def newDataset = new DefaultCategoryDataset()
    modules.each{ m ->
      def dsCall = attributes.call(m)
      newDataset.addValue dsCall[0], 0, dsCall[1]
    }
    this.dataset = newDataset
  }
}

Output
Let's run and look at sample output...



Click here for bigger images

Here is another script for Servlet Load Time

This one uses some intermediate groovy knowledge with attributes param taking a closure. It allows user to perform transformation on the MBean parameters. Below in the script it performs operates on long objectName attribute to get servlet name parameter.

jmx {
    server "service:jmx:rmi://localhost/jndi/rmi://localhost:1090/jmxconnector" {
      query "jboss.web:*" {
        findAll "j2eeType=Servlet" {
            chart{
              chartType="Bar"
              attributes={m-> [m.loadTime, m.objectName.find("name=([^,]*)"){it[1]}]}
              labels=["Load Time per Servlet", "Servlet", "Time"]
              options=[false, true, true]
              windowTitle="JBoss Servlet Processing Time"
              width=1200
              height=700
              orientation="HORIZONTAL"
              refreshRate=5000
              show()
            }
        }
      }
    }
}

and here is the output...



Click here for bigger image

Further you can,

• Add different type of chart support to the DSL - Bar, XY, trending, etc
  
• Use/Extend to work with different application server and/or application

Complete source code can be found at http://github.com/kartikshah/jmx-dsl

Blogged with the Flock Browser

Exploring Google Collections - Part 2

In Part 1, we focused on few classes of google collection. Here in part 2, we will expand on the simple scenario and work on DOM like tree structure scenario.

Consider a scenario of an external API provided by vendor. The API had tree like data structure, described by diagram below.


Now the problem was that the main data structure part of external API was POJO. Though it was a tree structure it did not provide any operations for traversal or find operations. Here is the watered down version of the component.

public class NodeComponent {
    private String name;
    private List<NodeComponent> children;
    private Map<String, String> attributes;

    public NodeComponent(String name)
    {
        this.name = name;
    }

    // Getters and Setters omitted for brevity
}

Let's see how google collection can be used to provide nice searcher methods. Consider following use cases:

• Find child nodes based on node name
• Find child nodes based on attribute name and value
• Find child nodes on composite criteria of node name and attribute name and value

Lets implement cases using google collections.

Case 1: Find child nodes based on node name
First we will define NodeNameCriteria implementing Predicate interface

public class NodeNameCriteria implements Predicate<NodeComponent>
{
    private String nameCriteria;

    public NodeNameCriteria(String nameCriteria)
    {
        this.nameCriteria = nameCriteria;
    }

    public boolean apply(NodeComponent node) 
    {
        return node.getName().equals(nameCriteria);
    }
}

Further, define NodeSearcher class which will represent current node and will provide all these helper methods

public class NodeSearcher
{
    private NodeComponent currentNode;

    public NodeSearcher(NodeComponent currentNode)
    {
        this.currentNode = currentNode;
    }

    public Collection<NodeComponent> findChildrenByNodeName(String name)
    {
        return Collections2.filter(currentNode.getChildren(),new NodeNameCriteria(name));
    }
}

Case 2: Find child nodes based on attribute name and value
Similar to first approach, this requires creating a Predicate implementation, let's name it AttributeNameValueCriteria

public class AttributeNameValueCriteria implements Predicate<NodeComponent>
{
    private String nameCriteria;
    private String valueCriteria;

    public AttributeNameValueCriteria(String nameCriteria, String valueCriteria)
    {
        this.nameCriteria = nameCriteria;
        this.valueCriteria = valueCriteria;
    }

    public boolean apply(NodeComponent component)
    {
        Set<Map.Entry<String, String>> entrySet = component.getAttributes().entrySet();

        Set<Map.Entry<String, String>> matchedAttrSet = Sets.filter(entrySet, new Predicate<Map.Entry<String, String>>(){
            public boolean apply(Map.Entry<String, String> entry) {
                return nameCriteria.equals(entry.getKey()) &&
                    valueCriteria.equals(entry.getValue());
            }
        });
        return matchedAttrSet != null && !matchedAttrSet.isEmpty();
    }
}

Case 3: Find child nodes on composite criteria of node name and attribute name and value
So far so good but the real benefit of defining the Predicate comes with the third case. In this case what is needed is to define a criteria which is composition of NodeNameCriteria and AttributeNameValueCriteria. We don't need to define third criteria implementation. Instead we will use Predicates compose method. Add following implmentation of findChildrenByNodeNameAndAttributeNameValue to NodeSearcher

public Collection<NodeComponent> findChildrenByNodeNameAndAttributeNameValue(String nodeName, String attrName, String attrValue)
{
     Predicate<NodeComponent> compositePredicate = Predicates.and(new NodeNameCriteria(nodeName),
                                                        new AttributeNameValueCriteria(attrName, attrValue));
     return Collections2.filter(currentNode.getChildren(), compositePredicate);
}

In simpler approach, one would have defined filter criteria as condition of if loop. But with implementing them as Predicate implementation, you can reuse criteria implementation and mix and match with help of following Predicates function and and or methods.

Blogged with the Flock Browser