Thursday, June 21, 2007

Fun with Code Hinting


Hi Folks! As Eylon said, I'm a summer intern at Adobe, working with Eylon and others to develop Flex applications. I'm not a blogger, usually, but there's a first time for everything.

Although I have a lot of experience programming, particularly in Java, I had never written anything in Flex or ActionScript before. Consequently, I had to get my feet wet somehow.
One of the applications my team is working on happened to need "code hinting" for some of its inputs. The idea is that, much like Eclipse or FlexBuilder, text could be suggested to the user based on what they were typing. I was tasked, as a sort of warm-up, with writing this little widget.

After some trials and tribulations, I came up with widget I call CodeHintingTextInput. Since it's moderately useful and fairly flexible/extendable, and because I ran into a few interesting issues along the way, it was suggested I put this up in a blog. So here I am, blogging it.You can see it in action here. To try it out, try typing "John" into the Author field and hitting Ctrl-Space, or typing "program" into the Interface field.

My widget in action

For those who are interested, I'm going to first go over some of the features of my widget, then talk about a few of the issues I faced, then present the source code along with an example.


Usefulness in different situations

As I thought about how to design my code hinter, I realized there are a number of different situations in which a code hinter is called for. I decided there were two important questions:

  • Is the input a single item, or is it a list? In other words, is the whole content of the text field the criteria for completion, or is only a part? A simple search box, for instance, would probably have only one item, as the suggestions would be whole search queries. But the "to" field in an email application might need the ability to auto complete many names, in this case separated by commas.
  • Do the contents of one suggestion depend on what the user has previously typed? The best example which answers "yes" to this question is Java or ActionScript class completion. If you type "m" in FlexBuilder, it doesn't offer you every class in the mx package, it offers you "mx." If you accept mx, then it offers you "controls," "components," etc. In other words, hints are only a level at a time. I decided that no code hinter would be complete without this feature.

I came up with a design which would allow someone to use this widget in any of the four situations that might result from answers to these two questions. I came up with two properties: listDelimeter and treeDelimeter. By default, these are both empty, resulting in functionality that would please someone who answered "no" to both questions above - in other words, a simple auto-complete feature, like ones offered by most web browsers.

Setting the listDelimeter to any number of characters makes typing those characters indicate the end of the current item. In other words, when matching suggestions, the widget will look back only as far as the last list delimiter.

Setting the treeDelimeter, on the other hand, tells the widget that, once it encounters a tree delimiter, to make suggestions which are the child suggestion of what was typed before the delimiter. That's probably not the clearest way to put it, but anyone who's familiar with any object-oriented IDE will recognize the applicability to class completion.

Some example applications:

  • A field that asks you to enter your name. Single item, no trees involved. Don't set any delimeters.
  • A field that asks for a list of participants in a meeting. List of items, but no trees involved. Set the list delimiter to ",".
  • A field that asks for an mathematical expression based on some variables. This might not look like a list, but it is. It just happens there are several delimiters, each with a special meaning. Set the list delimiter to "+-*/^" and any other mathematical symbols you wish to implement.
  • A field that asks for a comma separated list of ActionScript classes. Set the list delimiter to "," and the tree delimiter to ".".

Ease of Integration

I knew I would need to integrate this widget into our current code. Therefore,I wanted a component that could be integrated fairly transparently. For this reason, I chose to extend a TextInput widget. My goal was to be able to simply go through our code and change TextInput to CodeHintingTextInput, along with adding a few additional properties. Existing styles, properties, and event handlers would continue to work.


I promised myself I would offer as much configuration as possible. I keep coming up with new properties that make my widget even more flexible, so far I've added a number of options, such as when the hints pop up, whether whitespace is ignored, and whether matches are case-sensitive

Integration with Data Structures

My initial version only accepted a list of strings as a source for suggestions. If you wanted to auto complete classes, you had to enumerate each class, with its fully qualified package name, into an array and then pass that array to the widget. Similarly, if you had a collection of objects which contained completion data (say, a Person object which had a Name field), you had to extract the relevant info into an array first.

I realized at this point that the class structure I would be auto-completing was modeled as a tree structure, rather than a list of classes, and that it was totally infeasible to convert it to a list. So instead, I decided to come up with a generalized solution which would allow my widget to offer suggestion from any kind of data structure. This resulted in a very flexible but slightly complicated solution.

Basically, I needed to be able to extract two pieces of data from any object I was given. First, the actual text to suggest. I called this the "name" of the object. Second, in the tree case, I needed to be able to extract the object's children. I decided to accomplish this by allowing the user to specify getName and getChildren callback functions. These functions take in an object and return a string or an array, respectively. The user can define these functions themselves, giving them complete flexibility.

To sweeten the deal, I provided two default callback functions, which mimic the original behavior. The default getName function, for all purposes, simply returns object.toString(). The default getChildren function performs some text parsing on a string containing tree delimiters, and effectively chops off the first portion of the string.

Interesting Issues

When to Popup?

One of the biggest issues with any suggestion tool is when to offer the suggestions. You don't really want the user to have to ask for suggestions, because the whole idea is to save the user work. On the other hand, you don't want it popping up all over the place, annoying the user! Achieving a balance is key.

I certainly don't claim to have a magic solution. I studied the behavior of Eclipse and FlexBuilder for inspiration. Basically, I came up with the following ideas:

  • The user should always have the ability to summon the suggestion menu, and equally important, to dismiss it. I provided an Eclipse-like Ctrl-Space command to summon the menu, while Escape dismissed it.
  • In some cases, the user always wants the menu to pop up. So there's a property that can be set so if there's a suggestion, the menu will always pop up.
  • Assuming the user wants to summon the menu themselves, typing a list delimiter ends the current suggestion scope. If the user wants suggestions for the next scope, he or she should have to summon the menu again.
  • Tree-like data especially lends itself to auto completion. So typing a tree delimiter should automatically summon the menu. This approach is also embraced by Eclipse and FlexBuilder.

As we continue to use this widget, we might and probably will change our minds about all this…

Defeating Low-level Text Input Behaviors

As it turns out, the up and down arrow keys, which I wanted to use to make a selection in the list, function like Home and End keys in a single line text field. So, when you tried to make your selection, the cursor would jump to one end or the other, which was extremely annoying and no good at all.

I investigated various ways to block this behavior. I experimented with various permutations of key up & key down events, preventDefault & stopPropogation, and capture & bubble phases. Unfortunately experiments showed this is inherent behavior in the Flash Player that can not be blocked.
So, by this point, you may be wondering what the answer is. Unfortunately, there really isn't a good solution, which is rather unfortunate. So, like any good programmer, I resorted to the ugliest tool in my toolbox: the hack. By using key listeners, I realized I could save the cursor's position a split second before it moved, then could restore it later. So, my solution did the following: save the cursor position, start a one millisecond timer, then have that timer fire off a method which set the cursor position back to the saved position. I can hear the groans already...

CursorMovedEvent - Or Lack Thereof

When I tried to make the popup follow the cursor, instead of appearing in a static location, I realized I needed to be notified when the cursor changed position. Unluckily, there's no event for that. To compensate, I resorted to trapping all key events which might move the cursor, such as right, left, home, end, etc. The problem was, as I mentioned earlier, these events are fired before the selection changes. So instead of being able to set the popup to the current location of the cursor, I had to calculate ahead of time where the cursor was going to be and move the popup there. Very fun.


Here's a simple example of how to use this widget. It will generate an application that looks like this:

Basically, this application offers the user four fields as part of a "New Class" wizard, as might be used by an IDE. These four fields have differing levels of auto-completion, from none to a complete tree and list based completion model. Note that PathNode is a class I wrote which is not posted here. It represent a single node in a tree.

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="" layout="vertical" xmlns:local="*" horizontalAlign="center" creationComplete="init()">

<mx:Style source="mystyle.css" />


public function init():void

//create a list of possible authors
hintsource.addItem("John Doe");
hintsource.addItem("Jane Doe");
hintsource.addItem("Tom Nobody");

//create a list of possible classes
//as a list of strings

//greate a list of possible interfaces
//as a tree
var project:PathNode = new PathNode("project");
var common:PathNode = new PathNode("common");
var controls:PathNode = new PathNode("controls");
var base:PathNode = new PathNode("base");
var unused:PathNode = new PathNode("unused");
var viewable:PathNode = new PathNode("Viewable");
var styleable:PathNode = new PathNode("Styleable");
var bindable:PathNode = new PathNode("Bindable");
var removable:PathNode = new PathNode("Removable");
var unusable:PathNode = new PathNode("Unusable");
var sharable:PathNode = new PathNode("Sharable");




//Sample handlers who know how to handle a tree made up of PathNodes
public function getPathNodeName(obj:Object):String
var pn:PathNode = obj as PathNode;
return pn.getName();

public function getPathNodeChildren(obj:Object):Array
var pn:PathNode = obj as PathNode;
return pn.getChildren().toArray();

<mx:ArrayCollection id="hintsource" />
<mx:ArrayCollection id="hintsource2" />
<mx:ArrayCollection id="hintsource3" />

<mx:Panel title="New ActionScript Class" width="50%" height="75%" status="Please enter data" titleStyleName="heading2" verticalAlign="middle" horizontalAlign="center">

<!-- This input has no hints array, so behaves like a text input -->
<mx:Label text="Class Name" />
<local:CodeHintingTextInput minWidth="200" maxWidth="200"/>

<!-- This input has a data source but no delimiters -->
<mx:Label text="Author" />
<local:CodeHintingTextInput hints="{hintsource}" minWidth="200" maxWidth="200"/>

<!-- This input uses tree delimiters to specify a single class. Note it uses the -->
<!-- default handlers to parse a list of classes into a tree -->
<mx:Label text="Superclass" />
<local:CodeHintingTextInput hints="{hintsource2}" treeDelimitingString="." showWhenMatchesAvailable="true" minWidth="200" maxWidth="200"/>

<!-- This input uses tree delimiters and list delimeters for a list of classes! Also, -->
<!-- it uses custom handlers because its data source is a list of PathNodes, not strings -->
<mx:Label text="Implemented Interfaces" />
<local:CodeHintingTextInput id="treeguy" hints="{hintsource3}" treeDelimitingString="." listDelimitingString="," minWidth="200" maxWidth="200"
childrenFunction="getPathNodeChildren" nameFunction="getPathNodeName" showWhenMatchesAvailable="true"/>



Let's examine the four fields created. The first has no hint data associated with it, and behaves just like a text input. The second is looking for a single value, a name. In order to get the popup, the user must summon it by pressing Ctrl-Space. The third asks for a single class name. This is a tree, but is specified by passing a list of fully qualified names as a list of strings. Note that the showWhenMatchesAvailable property has been enabled, making the suggestion list pop up without a request.

The fourth field is by far the most complicated. It is a list of classes, so it has both list and tree delimiters. Furthermore, it is passed a list of PathNodes, rather than a list of strings. A PathNode is a simple tree node, with a variable number of children and the appropriate accessor and mutator methods. However, by default there is no implementation to deal with this, so the user provides his own simple functions to allow the widget to understand the data structure.

And Finally, the Code

The code, associated documentation, and the example discussed can all be downloaded here.

Guest blogger: Andrew Brindamour

Andrew has joined my team as an intern for the summer. :-)

Tuesday, January 30, 2007

Napkin skins, stage three: the (current) limits of skinning

After seeing what we can do with graphical and programmatic skins, it is time to explore what you can't do with Flex skins (yet). Unfortunately, you need to subclass ComboBox and DataGrid in order to get to the full napkin theme (click on the screenshot to see the app):

The changes at this stage are reflected in the MXML file (to see the complete file, select View Source from the app's context menu):
    <components:ScrawlGrid id="dg" width="100%" height="50%" dataProvider="{employees.employee}">
<mx:DataGridColumn dataField="name" headerText="Name"/>
<components:ScrawlComboBox width="50%" dataProvider="[Lorem, Ipsum, Dolor, Amet]"/>
[Note the use of <components:columns> instead of <mx:columns>.] We are now using the subclasses of ComboBox and DataGrid: components.ScrawlComboBox and components.ScrawlGrid.

ScrawlComboBox differs from ComboBox only in specifying a different dropdownFactory:
    private var _ddFactory:IFactory = new ClassFactory(ScrawlComboBoxDropdown); 
Subclassing ComboBox allows us to do two things: (1) specify a post-it skin for the dropdown list (in scrawl4.css):
    ScrawlComboBoxDropdown { background-image: Embed('assets/postit.jpg'); }
and (2) specify a cross-hatch pattern for rollover and selection in the dropdown list (in
    override protected function drawSelectionIndicator(
indicator:Sprite, xx:Number, yy:Number,
ignoredWidth:Number, h:Number, color:uint,
var w:int = unscaledWidth - viewMetrics.left - viewMetrics.right;
ScrawlUtil.drawCrossHatch(Sprite(indicator).graphics, w, h, color, 1.0);
indicator.x = xx;
indicator.y = yy;
Regarding (1): Since the background-image style is inherited, we could have simply created the style for List, which would have the same effect for this app, without subclassing ComboBox (since the default dropdown for a ComboBox is a List), but that would have meant that all Lists in the UI would have had a post-it background, which is not what we want here.

Regarding (2): Note that ScrollComboBoxDropdown subclasses List and overrides drawHighlightIndicator() and drawSelectionIndicator(). Flex makes it easy to skin rollover and selection in a List or DataGrid if all you want to do is change the color:
    roll-over-color: #E4E444;
selection-color: #C0C022;
However, if you want to draw any patterns in there, you currently have to subclass.

Looking at the code for ScrawlGrid, we see that, in addition to the cross-hatch pattern for rollover and selection, there are other things that require overriding DataGrid:

(1) Drawing the column separator: ScrawlGrid overrides drawVerticalLine() to add some randomness so that the lines are not exactly vertical. To get the full effect, we also need to add the following graphical skin as a DataGrid style:
    header-separator-skin: Embed('assets/header_separator.gif'); /* transparent */
Strange but true: the header separator is skinnable, but the column separator as a whole is not.

(2) Drawing a diagonal line fill in the header: ScrawlGrid overrides drawHeaderBackground() -- once again, flex makes it easy to specify a solid background color for the header, but you have to subclass in order to draw any patterns.

We now get to the most difficult part of creating this theme: drawing rollover and selection indicators in the DataGrid headers. In DataGrid, the drawing code for rollover and selection for headers (but not for the DataGrid's contents) is hidden inside mouseOverHandler() and mouseDownHandler(). So we need the hack of overriding mouse handlers in order to change a purely visual part of the app:
    override protected function mouseDownHandler(event:MouseEvent):void
headerRendererHack(event, getStyle("selectionColor"));
Note the call to super.mouseDownHandler() -- we can't just ignore the superclass' mouse handling code the way we did with the draw***() methods. headerRendererHack() draws the cross-hatch pattern in the renderer for the correct header -- but there's a twist to finding that renderer. It turns out that mouseEventToItemRenderer() will return the correct header renderer except for the case where the mouse is over the sort arrow's hit area. What we want in that case is the renderer for the header of the column being sorted. getColumnHeaderRenderer() finds that renderer for us.

That's it! (whew!) We now have reached the final version of the styled, skinned and tweaked Napkin app. Hopefully the tweaking needed at this stage will decrease over time as it is replaced with a more versatile use of programmatic and even graphical skins.

Thursday, January 25, 2007

Napkin skins, stage two: programmatic skins

We can build on stage one by using programmatic skins (click on the screenshot to see the app):

There were no significant changes to the MXML file: the only change is the reference to the CSS stylesheet. Looking at scrawl3.css (when you select View Source in the app's context menu), we see that the changes to the stylesheet are limited to styles that are specified as ClassReferences, for example:
    track-skin: ClassReference('skins.scrawl.ScrollTrackSkin');
These are references to programmatic skins written in ActionScript, which you can find in the source tree under skins.scrawl. The simplest one,, specifies the skin for the ScrollBar's track (but not for the thumb or the arrow). The drawing happens in the overridden method updateDisplayList() and, aside from a drawing a simple border around the track, does only one interesting thing:
    // fill with zero alpha: if no fill, mouse events are not recognized
drawRoundRect(1, 1, w-2, h-2, 0, 0x000000, 0.0);
I found that, without this line, clicking on the track does not move the ScrollBar thumb to the location of the click -- the mouse event is ignored, presumably because it doesn't hit any area drawn by the track skin. My solution for that was to add an invisible background for the track. I have also used this solution in my other programmatic skins, but I am not sure that this is the "recommended" way to address this issue.

Another simple programmatic skin is Someone with a different balance of graphical vs. coding skills would have probably created this as a graphical skin. For me, this was the simplest way to go. One advantage of using the programmatic skin is that you can make slight variations for up-/over-/down-skins within the same skin class. In this case, I have used conditions on the skin's name to determine the color and direction of the arrow, which allowed me to use the same skin class for all ScrollBar arrow skins:
    down-arrow-down-skin: ClassReference('skins.scrawl.ScrollArrowSkin');
down-arrow-over-skin: ClassReference('skins.scrawl.ScrollArrowSkin');
down-arrow-up-skin: ClassReference('skins.scrawl.ScrollArrowSkin');
up-arrow-down-skin: ClassReference('skins.scrawl.ScrollArrowSkin');
up-arrow-over-skin: ClassReference('skins.scrawl.ScrollArrowSkin');
up-arrow-up-skin: ClassReference('skins.scrawl.ScrollArrowSkin');
Programmatic skins also allow for some nifty tricks. The code in that draws the border of the ComboBox uses Math.random() to draw it differently every time:
    g.moveTo(Math.random() * borderThickness, Math.random() * borderThickness);
g.lineTo(w - (Math.random() * borderThickness), Math.random() * borderThickness);
g.moveTo(w - (Math.random() * borderThickness), Math.random() * borderThickness);
g.lineTo(w - (Math.random() * borderThickness), h - (Math.random() * borderThickness));
This helps give the ComboBox that ragged scrawled-on-a-napkin look: the border is different for the up-, over- and down-skins. If you play with the box dividers, you'll see that the ComboBox redraws itself with a slightly different border, even for the same skin. A similar principle is at work for the DataGrid's border (in, as well as for the cross-hatch pattern for the ScrollBar thumb skin (ScrollThumbSkin.drawCrossHatch()) and the ComboBox's down-skin (ScrawlUtil.drawCrossHatch() in the util package):
    // draw \\\\\\\\
for(i = -h; i < w; i += 6)
vstart = Math.max(-i, 0);
vdistance = Math.min(h, w - i);
vstartOffset = vstart + 3 - (6 * Math.random());
g.moveTo(i + vstartOffset, vstartOffset);
g.lineTo(i + vdistance, vdistance + 2 - (4 * Math.random()));

// draw ////////
for(i = 0; i < w + h; i += 6)
vstart = Math.max(i - w, 0);
vdistance = Math.min(h, i);
vstartOffset = vstart + 3 - (6 * Math.random());
g.moveTo(i - vstartOffset, vstartOffset);
g.lineTo(i - vdistance, vdistance + 2 - (4 * Math.random()));
Once again, programmatic skins allow us to call this code only for the ComboBox's down-skin while reusing the same class for all three skins:
    down-skin: ClassReference('skins.scrawl.ComboBoxSkin');
over-skin: ClassReference('skins.scrawl.ComboBoxSkin');
up-skin: ClassReference('skins.scrawl.ComboBoxSkin');
As one final touch, the DataGrid's border looks like it was drawn with a marker, with varying pressure on the marker producing color variations along the way. This effect is accomplished by specifying the following gradient type in ScrawlBorder.updateDisplayList():
    g.lineGradientStyle(GradientType.LINEAR, [borderColor, borderColor], [1.0, 0.5], [0, 255],
null, SpreadMethod.REFLECT);

Wednesday, January 24, 2007

Napkin skins, stage one: CSS styles and graphical skins

Quite a bit can be done using only CSS and graphical skins (click on the screenshot to see the app):

If you look at the source -- you can see the source files by selecting View Source from the app's context menu -- you will see that the MXML file only contains one new line:
    <mx:Style source="scrawl2.css"/>
The rest of the work is done by the CSS stylesheet. The font has been changed to AEnigma Scrawl (the font file is in the source tree under /assets, as are the images mentioned below) using the font declaration
font-family: scrawlFont;
src: url("assets/aescrawl.ttf");
font-weight: normal;
which is referenced in most cases by setting the style
    font-family: scrawlFont;
except for the DataGrid headers, which use the .Scrawl style declaration and
    header-drag-proxy-style-name: Scrawl;
header-style-name: Scrawl;
The background uses the napkin image from the Napkin L&F itself:
    background-image: Embed('assets/napkin.jpg');
and stretches to fill the background:
    background-size: '100%';
But, in order for the DataGrid to show us that background, we need to disable the alternating row colors:
    alternating-item-colors: ClassReference(null);
The DataGrid headers are still opaque; there's nothing we can do about it using only CSS as far as I know, but by the end of the third stage this will also be fixed.

Other graphical skins that are embedded in this version: the horizontal and vertical dividers and the DataGrid's sort arrow:
    divider-skin: Embed('assets/box_divider.gif');
sort-arrow-skin: Embed('assets/sort_arrow.gif');
The stylesheet also sets up the color styles for the components, as well as the border thickness for the DataGrid, in preparation for further skinning in the next stages.

Note: If you are looking to reuse the assets shown here, check out assets/aenigmascrawl.txt and assets/napkin.txt in the source view for the terms governing the use of AEnigma Scrawl font and the napkin (and postit) images. Thanks are due to Brian Kent and Ken Arnold for clarifying these terms for me and for creating these in the first place! The box divider and sort arrow are my creations and are, sadly, close to the limit of my graphical design ability. If you want to use either of those, knock yourself out. ;-)

Napkin skins in Flex

How easy is it to skin a Flex application? As an experiment, I tried to create something that looks like the Napkin L&F for a simple Flex app. (The idea of the Napkin L&F is to have your UI look like it was scrawled on a napkin -- which might be useful if you wanted people to get the fact that your mock-up was just a mock-up and not a fully functional app.)

The application is a slightly modified version of the DataGrid sample from the Flex 2 Component Explorer. Here are the "before" and "after" screenshots (click on the screenshot to see the app):


So, was it easy? I'll break it down into three separate stages. The first stage involves adding CSS styles and graphical skins to the app; the second stage involves using programmatic skins; the third stage deals with subclassing Flex framework classes in order to achieve fine-grained control of the look-and-feel.

Thursday, January 18, 2007

Hi there!

Hi! This blog is a place for me to post my thoughts on Flex. Hopefully I won't ramble too much... :-)

A bit about me: I joined Adobe a couple of months ago. Although I am technically part of the Flex team, I'm not really working on Flex right now, and my Flex coding skills are still pretty rudimentary. My background before joining Adobe is mostly in writing desktop Java apps.