Getting Started with Extjs

I was recently approached by an internal co-worker within Accenture about our teams experience with Extjs. Out of 150,000+ employees in Accenture he was able to find us through Yammer, a free private twitter-like service for companies. Originally I was just going to respond to him via email after our phone conversation, but that would only benefit him. Instead I thought it might be a good idea to share that experience on this blog.

My current team has been using Extjs for about 2 years. I don't consider myself an expert nor do I actually like developing in a language that needs to work in multiple browsers. We started out with version 2.x and late last year converted our projects to version 3, which was no easy task.

I will say Extjs has impressed this Java developer. Their documentation and examples are excellent and feedback on the forums is great. Most of my team didn't have any prior heavy javascript backgrounds and all have been able to come up to speed quickly. Right now I think the biggest drawback is the GPL licensing of version 3. I understand why they did it, but doesn't mean I have to like it.

Extjs Resources

• Examples - Extjs comes with a nice library of great examples on all kinds of things you can do with Extjs. All the examples also come with the download. One of the neatest examples is the Advanced Grid Filtering. One of the main reasons we upgraded to version 3 was for this feature.
  
• API Documentation - The javadoc for extjs. This is the most important and most referenced documentation for any extjs developer. Keep in mind this only shows the most recent version. If you want the API documentation for previous versions, you'll need to download that version of Extjs. The download includes the same API documentation. For local installation or offline reference, there is also a very nice Adobe Air app (see download page).
• Forums - The forums are very active and we have received a lot of help. We also have purchased the premium support and overall I'd say it was worth it. Especially since we no longer have that goto javascript guru anymore (yeah you know who you are).
• Community Plugins - It's not huge but between the examples noted above and the community plugins we have been able to reuse a lot and create some neat stuff pretty fast.
• Blogs - I'd recommend subscribing to the official Extjs blog as well as this search related feed from DZone. And, just because I think it is cool, here is Google Reader bundle I created that captures everything I tag with extjs. Finally, here is a search on my blog about articles I have written about Extjs.
• Books - According to Amazon there are multiple books out right now about developing with Extjs. I have "Learning Ex JS" by Shea Frederick.

Miscellaneous

• Javascript/CSS Consolidator - One of the best moves we made was investing in a javascript/css consolidator. Props go out to Joe Kueser for doing this for us. After a few months of development, our application started to grind to a slow crawl as several (80+) javascript and css files where being downloaded. Using the jawr framework we reduced that to around 20 GET requests which improved performance dramatically. Not only does it combine multiple files into one but it also minifies them (removes comments and spaces, etc) and supports gzipping them. I have been very impressed with jawr. It has exceeded our expectations and I would recommend it to anyone.
• Development Environment - No web developer would be complete without Firefox and Firebug. But for those occansional nasty IE issues I haven't found the perfect solution. In the past I have had some luck with jsdt. Currently I am using IE8 in virutalbox in IE7 mode with it's built in Developer Tools. Although not firebug, it's the closest I have come to find. For those really bad IE6 issues I have used the Internet Explorer Developer Toolbar which beats scattered alerts in your code and is better than nothing.
• Extjs version 2.x or 3.x - There has been a lot of history surrounding the licensing strategy so I won't go into all of that. Just know that version 2.x is LGPL and can be used or embedded into your applications for free. Version 3.x is GPL which means if your project isn't some internal IT app or also GPL you need to purchase developer licenses.
• Public Uses - Here are just a few well known sites I have noticed using Extjs: Quicken Online and Dow Jones Industrial Index.
• jQuery Integration - In case you want to combine jQuery and Extjs you can by using the extjs-jquery-adapter available in the download. I haven't really used jQuery a lot but from what I have read and heard, it's a great javascript library that supports animation and DOM querying. You can do these things in Extjs, but jQuery looks like it might do it better and cleaner. I think Extjs excels in providing out of the box prebuilt and customizable Widgets/Components like the Grid (Table).
  

Getting started with Grails and Extjs

This article describes how to get started using extjs with your grails app. Since the plugin is deprecated because of the GPL license fiasco, I decided to write my own simple grails script to handle installing extjs for the team for 2 main reasons:
1) Prevents me from having to commit 500+ files into SVN for every new version
2) Makes it easier to upgrade to newer versions of extjs in the future

We have 2 existing internal applications that use extjs and every upgrade I am kicking myself for not using maven to extract the extjs zip file. So with our new grails app I wanted to not make the same mistake.

Install Extjs
1) Download your choice of extjs. Note as of version 2.1 extjs is under the GPL license. Meaning if your project isn't open sourced under the GPL or an internal company app then you need to use the 2.0.x versions (2.0.2 is the latest). In our case I downloaded version 2.2.

2) Copy ext-2.2.zip into your grails plugins directory

3) cd into your grails application

4) Add the zip file to svn (or whatever): svn add plugins/ext-2.2.zip

5) run grails create-script install-extjs

6) Add the script to svn: svn add scripts/InstallExtjs.groovy

7) Modify the InstallExtjs.groovy script and add the following GANT code.

8) run grails install-extjs

9) Exclude the unziped ext directory: svn propedit svn:ignores web-app/js Exclude the folder ext.

Test it out
Now that you have extjs installed you can copy one of their examples into the grails web-app directory and update the links.

1) Open up the ext-2.2.zip file again and extract the array-grid.html and array-grid.js files from the examples/grid folder to the grails web-app directory.

2) Modify the array-grid.html file. Update the relative links for css and javascript by replacing ../../ with js/ext/. For example, href="../../resources/css/ext-all.css", should now be href="js/ext/resources/css/ext-all.css"

3) Open up the array-grid.html file in your browser.

Now you have extjs installed with a simple example on how to use it. Next I would like to create a grails controller that returns JSON to populate a simple grid.

Learning with Grails: Security, Extjs, REST, Spring Insight

Unfortunately, security and performance are often postponed early in the development process and are surpassed for new functionality. Reasonable justification usually includes cost and time. A few of the real issues I think are difficulty and lack of experience. I have to admit I am no security or performance expert, and implementing both early on would be time consuming and difficult. I've always heard security done after the fact is never as good if it's done in the beginning. So I wanted to share my experience using the spring-security (acegi) plugin in grails. Not only did I learn a lot about security, but also how to add support for REST Services and integrate Ajax clients using Extjs, how to enable caching, and inspect the performance of my app using Spring Insight that comes with the Springsource tc server developer edition.

Table of Contents

• Spring-Security (Acegi)
• REST Support for Extjs Ajax clients
• Spring Insight and Caching
  

Spring-Security (Acegi)

Building authentication and authorization into your app in the beginning can be difficult, but with the spring-security plugin and grails it's super easy. There are other security plugins for grails, but I decided on spring-security/acegi since our team has contemplated using it on our project several times and it seems to have a pretty good history. This isn't a how to guide on using the security plugin, the documentation is pretty good. However, I do want to share some unexpected things I ran into and how I arrived at some of my conclusions.

When applying the security capabilities, I applied Test-Driven Development (TDD) to test my assumptions and changes. After I installed the plugin, I got started right away creating Users, Roles, and Requestmaps in Bootstrap. Requestmap is the domain model mapping roles to URIs. I prefered this method initially because I wanted to persist this information to the database. First I wanted to lock down the ability to delete a model I called Event. Here is a sample Bootstrap to accomplish it:

In this first test I created an admin role, mapped the admin role to /event/delete, and then added a new user to the admin role. This worked as expected but had a pretty big security hole (see Unexpected observations using spring-security plugin). The issue was unauthorized users were still able to delete events using the edit form. Underneath, grails submits to the index action and the controller handles directing the request to the delete action in the controller bypassing the security created by my Requestmap. I could hide the Delete button on the edit page, but malicous users could still expliot this hole.

So instead of using Requestmap, I annotated my controller actions. This method can properly handle the use case above and deny unauthorized access to the delete action.
I also learned that users have to be associated with a Role, otherwise they are unable to log in (see post). This seemed rather annoying since I was expecting to use the predefined roles: IS_AUTHENTICATED_FULLY, IS_AUTHENTICATED_REMEMBERED, and IS_AUTHENTICATED_ANONYMOUSLY without having to associate all my users with roles. Two suggestions by Burt where to extend a Base model class that supported a default Role for all Users, or extend the GrailsDaoImpl class to remove the role requirement for users.

In summary, it seems best to annotation your controllers instead of using Requestmap and keep in mind that by default all users need to be associated with a role to login into your application.

REST Support for Extjs Ajax clients

Now that I have portions of my app locked down, I wanted to see how this effected Rich Internet Applications (RIAs) such as those that use Extjs. I wanted to answer 2 basic questions:

1. How can javascript clients remove admin functions like Delete buttons?
2. How could I modify my controller to support multiple clients that need HTML, JSON, or XML?

To test these questions, I downloaded and installed extjs into my grails app and created a simple grid based on the array-grid example in extjs. As part of this test I wanted to add a Delete button to the bottom toolbar, so that I could later enable or disable based on the users role.

I'm not super proud of how I contrived the roles and used them in javascript, but below is how I did it (If you were doing this for real, I'd make this a point of focus to come up with a better solution like creating a service that returned this information as JSON that could be called by an Ajax client).

Since there is no real easy way to get access to the users roles in a gsp, I set a bean in the controller that is used in the gsp. So in EventController I added the following grid action:
Then in a new grid.gsp I added the following:

Ext.onReady(function() {
    var roles = new Ext.util.MixedCollection();
    <% authorities?.each { %>
        roles.add("<%=it.authority%>");
    <% } %>
});

Then when I build the grid, I could eanble or disable the Delete button by doing:

roles.contains("ROLE_ADMIN")

That's pretty much how I disabled certain gui items that are restricted based on role. Again not very pretty, but effective.

Next, I wanted to populate my grid with real data from my controller. Fortunately for developers, grails excels in this area using convention over configuration using the withFormat content negiotation with URI Extensions. All I had to do was modify the controllers list action to support more than HTML responses:

def list = {
    params.max = Math.min( params.max ? params.max.toInteger() : 10,  100)
    def events = Event.list(params)
    withFormat {
        html { [eventInstanceList: events, eventInstanceTotal: Event.count()] }
        xml { render events as XML }
        json { render( [list: events] as JSON ) }
    }
}

Here I easily add support for XML and JSON clients while also continuing to support clients who want HTML. One thing to note, is I like naming arrays in JSON; that is why you see the render([list: events]) syntax. Without that, the JSON response looks like this

{[{"class":"Event".....

instead of my preferred way

{"list":[{"class":"Event".....

Now all the client needs to do is request the URI /event/list.json or /event/list.xml.

Here is my complete javascript that includes the example JsonStore and Grid definitions.

Spring Insight and Caching

Next, I really wanted to try out the Spring Insight capability that is now included in the Springsource tc server developers edition. This would let you see into how grails and hibernate are operating on your behalf per request. For many web applications, there are typically 2 bottlenecks that degrade performance: 1) number of trips from the client to server 2) number of trips to the database. Using the Spring Insight tool, developers can see what SQL is being executed and how long it took.

To get started with Spring Insight I followed their Getting Started Guide. Once I had Insight running (http://localhost:8080/insight), I next ran 'grails war' to create a WAR that I could deploy to tomcat (Note: if you do this multiple times, redeploy, you have to be careful with what you do in your apps Bootstrap class. I was doing a lot of inserts that caused my WAR to fail deployment because by default when running 'grails war', grails defaults to the production environment which uses an update file database. So my Bootstrap was trying to add roles and users that already existed and that caused deployment to fail).

The WAR deployed fine and I was up and running watching my apps performance metrics in the Insight dashboard as I moved around in my app. What I noticed first was unexpected multiple JDBC calls when I thought caching was enabled by default in grails.


What I learned was you have to enable caching on a per domain or query basis. So I added the following to my Event domain model

static mapping = {
    cache true
}

rebuilt and redeployed the war and was able to see fewer JDBC calls proving that my cached domain was working. I then added a new Event, viewed all events, and saw the extra select call since the cache was purged. Pretty impressive!

Next I wondered if spring-security was caching all the user and role information by default like it advertised. If not that could be a huge performance issue. Spring Insight was also able to prove that it was caching its results.

Overall, I learned a lot about grails, security, REST, extjs, caching, and spring insight and grails was the perfect platform to prototype these concepts in preparation for real production use.

How to Change Extjs PieChart Colors

When using Sencha's, or extjs, charting capability, most likely your going to want to change the default color scheme. I was faced with this issue today and it's not documented as well as you'd expect. I had to piece together a few articles and I'm still not 100% it's the right way as it's using an undocumented config option. But I wanted to document how I did get it to work to same others some time.

Sencha is very well documented and their examples are great. Here is the Pie Chart example we are going to be updating (using version 3.2.1). For those that don't know, extjs charts are based off of Yahoo's YUI charts which is also requires Flash. Not only is it pretty simple to create charts with extjs, but we already are using extjs so we decided to prototype some charts using it.

Here is the code that produces a simple basic PieChart:

new Ext.Panel({
    width: 400,
    height: 400,
    title: 'Pie Chart with Legend - Favorite Season',
    renderTo: 'container',
    items: {
        store: store,
        xtype: 'piechart',
        dataField: 'total',
        categoryField: 'season',
        extraStyle:
        {
            legend:
            {
                display: 'bottom',
                padding: 5,
                font:
                {
                    family: 'Tahoma',
                    size: 13
                }
            }
        }
    }
}); 

This produces the following PieChart.

To change the default colors provide a series config option:

new Ext.Panel({
    width: 400,
    height: 400,
    title: 'Pie Chart with Legend - Favorite Season',
    renderTo: 'container',
    items: {
        store: store,
        xtype: 'piechart',
        dataField: 'total',
        categoryField: 'season',
        series: [{
            style: {
                colors: ["#ff2400", "#94660e", "#00b8bf", "#edff9f"]
            }
        }],
        extraStyle:
        {
            legend:
            {
                display: 'bottom',
                padding: 5,
                font:
                {
                    family: 'Tahoma',
                    size: 13
                }
            }
        }
    }
});

The chart it produces may not look the most appealing but at least we figured out how to change the colors.

Pie Chart with New Colors

Again, I'm not certain this is the best way to accomplish this as the series config option isn't documented in 3.2.1. But by piecing together this PieChart question and this article, I was able to figure it out.

How to simulate onblur event for a Panel in Extjs

Here is a simple trick in Extjs I borrowed from Ext.layout.BorderLayout that simulates listening for the onblur event of an Ext.Panel. This trick is necessary because unfortunately Ext.Panel does not natively support registering for the onblur event (or at least I don't know how). We typically use this trick when we show a Panel and want to remove it when the user clicks off, or to state it differently, when the Panel looses focus. This is very similar to how menus work.

Step 1: Register for mousedown event

this.panel.show();
Ext.getDoc().on("mousedown", this.handleDocMouseDown, this);

First, we show the panel. Here it is assumed this was created previously. Second, we use Ext.getDoc() to return the current HTML document object, and then we register for the mousedown event. The second parameter, this.handleDocMouseDown, is our function we want called when this event is fired. Notice that we register for the mousedown event as opposed to the click event. This is intentional because listening for the click event will actually call this.handleDocMouseDown before your ready because a click involves both mousedown and mouseup.

Step 2: Handle the event

handleDocMouseDown : function(e) {
if (!e.within(this.panel.getEl())) {
   this.panel.destroy();
   Ext.getDoc().un("mousedown", this.handleDocMouseDown, this);
}
}

In our second step we first check to make sure the user didn't click anywhere in the Panel that was just displayed. If it wasn't then we destroy the panel and unregister the mousedown event.

RSS, Lucene, and REST

Sorry for the horrible title. I struggled trying to come up with a worthy title, but after a few minutes I decided to not let perfection get in the way of good.

My team has recently worked on a new feature I am pretty excited about: adding support for RSS/Atom in our application. I know your thinking so what. It's not really the what I am excited about but the how. What I'm really excited about was how the story was defined and implemented.

Approach
We had the simple requirement from a newer customer to provide an RSS feed for newly created items. This actually wasn't the first time for this requirement. We prototyped a similar capability a long time ago using OpenESB and the RSS BC, but for multiple reasons it just didn't work out.

So our first decision had to answer: how we were going to implement it......again, but better. Before the sprint began, a few of us got together and hashed out a potential solution: how about we use the Search REST Service, which is backed by Lucene, to support Advanced searches and return RSS?

Why does this excite me so much? To understand that I need to explain our application at a high level. It's a completely javascript-based application using ExtJS (now sencha), backed by REST Services using Jersey. Consequently, we have a lot of REST Services. Right now those REST Services support returning XML or JSON using a custom Response Builder we have created internally.

I'm excited because this single user story could have a huge improvement on the entire system:

1. If we modified the Search Service to return RSS, then all our REST Services could support RSS.
2. The REST Service would now support Advanced searches. Previously, it only really supported basic keyword searches.
3. Any search they perform could now be subscribe to via RSS.

Implementation
I'm not going to go into every detail on how it was done. I wasn't even actually the one who implemented it (see Matt White. He did a fantastic job.). We did have one major hurdle we had to overcome, and that was how to index items to enable advanced searches like Status=New.

Previously this wasn't possible given how we were indexing our items. We were basically indexing the item by building up a large String containing all the item information like the following:

import org.apache.lucene.document.Document
import org.apache.lucene.document.Field

def Document createDocument(item) {
    Document d = new Document()
    
    doc.add(new Field("content",
        getContent(item),
        Field.Store.NO,
        Field.Index.ANALYZED))
        
    return d
}

def String getContent(item) {
    def s = new StringBuilder()
    
    s.append(item.getTitle()).append(" ")
    s.append(item.getStatus()).append(" ")
    s.append(item.getPriority()).append(" ")
    s.append(item.getDescription()).append(" ")
    
    return s.toString()
}

The problem with this is performing a search for "New" would have returned any item with a status of New as well as any items that contained the word New. The solution was to just add another Field to the Document.

doc.add(new Field("Status",
    item.getStatus(),
    Field.Store.NO,
    Field.Index.NOT_ANALYZED));

Now the Search Service could support advanced searches like: Status:"New". You should put the value in quotes in case the value contains spaces (ie Status:"In Progress"). And since Lucene is so powerful, it also means the follow search would work: Status:"New" AND Priority:"High" AND "Hurricane". Now users have the freedom to subscribe to a near limitless amount of RSS feeds based on Advanced Searches.

Start to Finish
I think there were several reasons why this story was a success in my eyes. Most importantly where the two really smart co-workers who worked on it: Matt White and Chuck Hinson. All three of us knew of this user story ahead of time and we were able to discuss it technically days before backlog selection. This allowed us to brainstorm some ideas. Once we narrowed it down, we spent some more time separately looking into the code to find out the level of difficulty and if Advanced Searches like Status:New would be possible. Overall, together I'd say we spent 3-4 hours doing the preliminary work. Doing that preliminary work I think really enabled us to give a proper WAG for the story.

I really can't speak for how the development went (I was at Disney World for 10 days with the family), but I was really impressed with the tests Matt wrote. He wrote a number of unit tests making sure advanced searches worked and basic searches still worked. On top of that, he wrote an overall functional test using HttpBuilder executing the REST Service just as our javascript client would.

Finally, once the main work was finished, we uploaded a diff file to our internal instance of Review Board. From there I was able to perform a peer review where we found a minor bug in the changes.

Summary
I am sure it's not an original idea, but I thought it was a fun User Story that hopefully will provide a lot of value beyond what was originally estimated. Ideally, this might help others who are in similar situations.

Testing REST Services with Groovy

For awhile now, RIAs (Rich Internet Applications) have rapidly started replacing traditional server-side web applications (JSP, JSF, etc). Typically, these flashier sites are created using Flex or javascript libraries like extjs or yui. At the heart of these sexy applications live simple REST services that return JSON or XML.

Testing these REST services should be made a high priority for several reasons:

1. It's the contract between the client and server. Breaking that contract should be avoided.
2. Your application may not be the only client. Once external parties start consuming your REST services you'll want tests in place to ensure you don't break their clients.
3. To validate the response is well-formed XML or properly constructed JSON
4. Valuable black-box test, testing the entire server-side stack starting from the REST layer going all the way down to the DAO layer to the database.

So, what's the easiest way to test REST services? For awhile now I have been combing the internet for the best tools to accomplish this goal, since I wasn't going to do it in pure Java, and I think I finally found the right combination using Groovy and HttpBuilder. Groovy because its super easy to parse XML and JSON, and HttpBuilder because it's a great wrapper for the popular Apache Commons HTTP Client library.

Now let's say you have a REST service at the URL http://localhost:8080/myapp/api/books that returns this JSON:

{"books":[
  {"name":"Being Mad","author":"Kit Plummer"},
  {"name":"Clown Life","author":"Josh Hoover"},
  {"name":"You Miss Me","author":"Ron Alleva"},
  {"name":"Talk to me Goose","author":"Jeff Black"}
]}

This is how simple it is to write a test in Groovy using HttpBuilder:

import groovyx.net.http.HTTPBuilder
import groovyx.net.http.Method
import static groovyx.net.http.ContentType.JSON

class BooksTest extends GroovyTestCase {
  def void test_get_all_books() {
      def http = new HTTPBuilder("http://localhost:8080")

      http.request(Method.valueOf("GET"), JSON) {
          url.path = '/myapp/api/books'

          response.success = {resp, json ->
              json.books.each { book ->
                  assert book.name != ""
              }
          }
      }
  }
}

To me the major advantage to this approach is being able to traverse the JSON response like I would in javascript. If the response was XML it be just as easy too.

The only two remaining items you would need would be adding the gmaven plugin to your pom and httpbuilder as a dependency.