Introducing Protowidget
May 29th, 2006I’d like to introduce a new JavaScript/Ajax framework that my company is creating. We’re calling it Protowidget because it uses Prototype and does things with widgets. And yes, we know it’s 2006 and this is something like the 800th Ajax framework released this year. We think this one is new and different, though (of course, every parent thinks their baby is beautiful). It is being created out of real needs while building applications for customers and is helping us create better stuff. Please note that this is just a preview. Not everything works like it should or has been tested to the degree we would like. In particular, under certain circumstances, the library leaks memory. We know what needs to change to fix this but haven’t gotten to it yet.
Motivation
The best tools are born out of pain. With regard to existing Ajax tools, here is a brief list of some of the pain we felt and wanted to address:
- ID-madness: Some frameworks (ie. Dojo) attempt to avoid this, but most make the primary coupling point between JavaScript and HTML be the ID attribute. While many have shown this to work, it is somewhat offensive to have only one flat namespace. This becomes an especial problem when trying to create repeating sections or scoped, re-usable components. Since the namespace is flat, it also causes problems when one component needs to logically interact with others.
- Non-standard UI construction: At the end of the day, HTML+CSS is a pretty good tool for laying out web-based UIs. We can certainly come up with better representations, but they are rarely as rich and they are typically not designer friendly. Despite a lot of other great ideas, I consider this to be OpenLaszlo’s primary failing. Other frameworks which take the Swing-esque approach of coding the UI programatically by assembling objects are even weaker.
- No client side model: Years ago someone co-opted the long standing MVC design pattern and claimed that having servlets and views somehow conformed to the pattern. Maybe it technically does, but it is certainly not in the spirit of MVC as seen in real client-side apps. Today, we have an interesting inversion going on where most people think that the web-MVC (type II or whatever) pattern is classical-MVC. We need to be applying some classical-MVC principles to our client-side interfaces. Otherwise, we will end up with as much spaghetti as can be found in an old-school VB6 app. OpenLaszlo and Flex with their pervasive property bindings provide the building blocks for this type of development. Few others do.
- Separating form from function: This is the holy grail, and likely no tool will ever get 100% there. The idea is that the visual layout and characteristics should be specified out-of-band from the code that manipulates it. A good framework should have well-defined intersection points where the visual bits mix and interact with the programmatic bits.
In the end, we really liked the simplicity of Prototype, the widgets/templates from Dojo and the event-based property binding support from OpenLaszlo/Flex.
Example
Here is a link to one of the Protowidget test pages. It’s not much to look at and certainly doesn’t do anything that can’t be done elsewhere. The magic is in how it is done, and that is what the rest of this post will focus on. http://www.rcode.net/stage/trunk/protowidget/test/bind_test.html
This current version has been verified on Firefox 1.5 and Internet Explorer 6.
Widgets
As the name implies, Protowidget is based on widgets. While some widgets are provided (and more will come in the future), Protowidget’s primary focus is on providing a rich environment for constructing complicated widgets.
A widget is always bound to an HTML element. This will typically be a div, but can really be anything (depending on the widget). We take the Dojo approach of leveraging custom attributes in order to do all of the wiring. This has the advantage of letting us bypass the ID-madness when it comes to hooking widgets to DOM elements. Here is an example:
<span pw.type='Text' pw.text='#{`[currentSection].model.title`}'></span>
This HTML declaration tells Protowidget to bind the span to the built-in Text widget. It further sets up a property binding so that the text property will be bound to the value of model.title on the current section (this example came from the Accordian title-bar template so the currentSection reference is an Accordian.SectionWidget in this context).
Hierarchy
Widgets exist in a hierarchy of parent-child relationships that typically mirrors the HTML DOM elements that they are defined on. Each widget has a parent and may have an id property. A widget may also be attached to another widget (most likely its parent) by name. All of these techniques work together to create a navigable tree of widgets. Unlike the global IDs on HTML elements, Protowidget id’s are scoped such that they can only be resolved by children in the hierarchy. That is, referencing a widget by id from a child widget is equivilent to saying “Give me the widget arbitrarily up the parent chain with a given ID.” The top-level widget is a special singleton which logically is defined on the body element and has an id of “root”.
Properties
The Widget class extends the built-in Component class which is responsible for managing properties. Properties formalize the mechanism for changing an Object’s state by providing getters/setters, change event support, and hooks that are called when values are set or read. Every Component sub-class will have a getProperty and setProperty method which are the primary entry points for generically manipulating properties. There are convenience facilities available for declaring appropriate getters and setters for properties as well.
Attributes
In addition to properties, widgets also maintain the concept of attributes by exposing the methods getAttribute and setAttribute. These methods define the mechanism for navigating to other properties within the widget tree. Arbitrary parts of the tree can be referenced by specifying an attribute path such as ‘a.b.c’. There is a special escape in the attribute syntax: If a component of the path is surrounded by square brackets, it will resolve to the parent with the given id (between the brackets). While a similar effect could be achieved by using a correct number of parent references (ie. ‘parent.parent.parent.textField’), using the ID notation is more straight-forward: ‘[commonParent].textField’.
Bindings
Bindings represent the mechanism whereby any property can be bound to any other property in the tree. If the value of the source property (or the path to the source property) changes, the bound target property will be updated. This has already been illustrated by the Text example above. As with OpenLaszlo, the binding is done with a special syntax, called a Binding Expression in Protowidget. This is done by specifying the value of a property with the syntax:
#{someJavaScriptExpression}
where someJavaScriptExpression is any legal JavaScript expression. The trick that makes it so powerful is the special escape sequence using the backtick characters (`). Any sub-expression within the JavaScript expression may have a backtick reference. The attribute path to another part of the tree can be specified within the backticks and its value will be placed into the expression at evaulation time. The entire binding will be evaluated when first established and whenever the value or path of a contained backtick expression changes.
Here are some examples:
#{`[root].product.name`}- References the product.name property on the root widget.#{`[commonParent].position` * 50}- Evaluates to the position property on the commonParent parent multiplied by 50.
Special Support for Element Attributes
The DomWidget class (which most general-purpose widgets will extend) has special support for initializing properties on its instances. Any attributes defined on the attached HTML element which starts with ‘pw.’ and is not a standard Protowidget attribute will result in the corresponding property being set on the widget (as was illustrated with the pw.text HTML attribute above). These can either be literal Strings of binding expressions.
The DomWidget class treats style attributes specially by performing some conversions (ie. from numbers to pixel units) and allowing them to be specified on the element with HTML attributes of the form ‘pw.style.left’. It would be silly to define constant styles this way, but it is very useful for binding styles. For example:
pw.style.left="#{`[commonParent].position` * 50}"pw.style.color="#{`[section].status`=='success' ? 'green' : 'red'}"
Templates
When doing JavaScript programming “in the large”, it quickly becomes necessary to break visual parts out of the main flow. Protowidget defines a template mechanism whereby external HTML sources (which can contain widget declarations) can be loaded and instantiated within an application. We will not go into this facility in depth here, but suffice it to say that it forms the basis for some of the other capabilities presented later.
Models
Protowidget defines two general purpose model classes which may be extended as necessary to define intra-application models. By attaching these non-visual models to the root widget as properties, the model properties can be bound to any widget property in the application.
Model.Data
The Model.Data class is intended to be a generic container for properties. The intent is that it will provide advanced serialization/deserialization support for synchronizing with a server.
Model.DataList
The Model.DataList class provides an event wired List/array abstraction. It defines the following methods: getLength, itemAt, add, remove, and removeAt. In addition, it defines an ‘onlist’ event which is fired any time the list is structurally modified.
DataListRepeater
Protowidget defines an abstract BaseDataListRepeater class which provides the low-level machinery for keeping an arbitrary Model.DataList (specified by setting the ‘list’ property) in sync with visual widgets that represent each list item. The algorithm is efficient and can handle arbitrary changes to the backing list while only creating corresponding visual widgets for new entries.
The TemplateDataListRepeater extends BaseDataListRepeater so that a given template (specified by the template property) is instantiated for each element in the list. In the example, this is what is driving the numbered list that rotates every second.
Accordian
Protowidget defines an Accordian widget and related model. The widget is bound to the model and tracks all changes to the overall model or any of its sections. Internally, the accordian widget is a subclass of BaseDataListRepeater and illustrates the power of an abstract model-bound repeater widget.
An Accordian.Model consists of a Model.DataList of Accordian.SectionModel instances and an activeSection property. Each Accordian.SectionModel has the following properties: enabled, active, title, caption, owner and template. Most of these control the title bar. The template property specifies a template to be instantiated as the section’s content. All properties (including template) are bound, so that any changes take effect immediately. Display of the title bar is driven by a template and CSS class names have been defined such that all visual characteristics are externalized in the accordian.css file. All visual characteristics of the accordian can be changed by modifying the CSS file or the title bar template. This even goes so far as to allowing additional information or behaviour to be defined in the title bar by binding widget properties in the title bar template to user-defined properties on the SectionModel.
Here is an example of creating an Accordian model:
var accordian=new Protowidget.Types.Accordian.Model();
Protowidget.RootWidget.setProperty('accordian', accordian);
var section;
section=new Protowidget.Types.Accordian.SectionModel(
accordian, ‘Section 1′, ‘In Progress’);
section.setTemplate(’user!section1.html’);
accordian.sections.add(section);
section=new Protowidget.Types.Accordian.SectionModel(
accordian, ‘Section 2′, ‘Complete’);
section.setTemplate(’user!section2.html’);
accordian.sections.add(section);
section=new Protowidget.Types.Accordian.SectionModel(
accordian, ‘Section 3′, ‘Success’);
section.setTemplate(’user!section3.html’);
accordian.sections.add(section);
And here is how an accordian would be instantiated based on that model:
<div pw.type='Accordian.Widget'
pw.model='#{`[root].accordian`}'
style='width: 75%; height: 300px; overflow: hidden;'>
</div>
Logger
A Logger library is also included with Protowidget (see src/logger.js). We intend to break this out separately because it has no ties to Protowidget. While looking at the example, press ‘ALT-L’ to see the log window (if it doesn’t work, click somewhere in the document and try again to make sure the document has focus). The logger will automatically popup on errors. The primary methods to interact with it are Log.debug, Log.info, Log.warn and Log.error. Each of those methods also exposes an ‘enabled’ property for checking whether that log level is turned on. For example:
if (Log.debug.enabled) Log.debug("Hello World");
This would typically only be used if generating the log message was an expensive operation.
Final Thoughts
Releasing this work to the public is really just a preview at this point. Protowidget does things differently than a lot of other libraries, and we think it does them better. Please let us know what you think. Either post comments to our forums or email us personally.
The current snapshot of the source is here. Since the template system uses XmlHttpRequest, any examples must be run within a web server. There is a ruby script in the root directory that will start a web server to serve from the current directory (by default on port 2000).
Introducing Protowidget
May 29th, 2006I’d like to introduce a new JavaScript/Ajax framework that my company is creating. We’re calling it Protowidget because it uses Prototype and does things with widgets. And yes, we know it’s 2006 and this is something like the 800th Ajax framework released this year. We think this one is new and different, though (of course, every parent thinks their baby is beautiful). It is being created out of real needs while building applications for customers and is helping us create better stuff.
Head over to the dedicated page on this blog to find out more.
About
May 22nd, 2006Hi, my name is Terry Laurenzo and I own Redcode Technologies. This blog is mostly about technology and contains a mixture of work and non work-related content.
Setup Rails
April 26th, 2006I always seem to miss a step when installing Rails on Fedora. Here are the steps:
Setup Rails (all steps as root)
———–
Install optional Ruby packages:
- ruby-rdoc
- ruby-ri
Download and install Ruby Gems:
- wget http://rubyforge.org/frs/download.php/5207/rubygems-0.8.11.tgz
- Extract archive and change into directory
- ruby setup.rb
Install Rails:
- gem install rails –include-dependencies
Installing Postgres support:
- yum install postgresql-devel
- gem install postgres
Firefox View Source Trick
April 19th, 2006I might be the last one on the planet to discover this one, but I doubt it, so I thought I’d share.
I’m used to right-clicking on a page and selecting “View Page Source” from the context menu. This has some advantages over the same command in the View menu because it allows you to narrow down which frame you want to view.
Last night, however, I accidentally selected some text and right clicked. I noticed that the familiar “View Page Source” menu item had changed to a “View Selection Source”. It seems that if an straight-forward subset of the document is spanned by the selection, the resulting view source window will only display that portion. If, however, the selection spans disjoint elements in the tree, the whole document will be displayed with the relevant pieces hilighted.
This in and of itself is really helpful for comprehending a complex HTML structure, but there is one more thing I noticed that is even better:
The output of the View Selection Source command seems to be the HTML representation of the current structure of the document as opposed to the raw text obtained at page load time. Put another way, the changes made via JavaScript to the HTML DOM are displayed by View Selection Source.
I know there is an extension that lets you view the live document structure but I’ve never installed it and this is an easy trick that seemingly produces the same result.
Getting dev tools working with XULRunner
March 17th, 2006I’ve recently had a revived interest in Mozilla XUL. Last month I saw a blog about the new XULRunner tool, so I downloaded it a few days ago for a closer look. The first thing that became pretty obvious was that the normal suite of tools that I’m used to working with in Firefox/Mozilla don’t really fit into XULRunner out of the box. These include things like the Extension Developer Extension, the Venkman JavaScript Debugger, and the DOM Inspector.
There are a few places that mention that it should be possible to get these things going, but I couldn’t find any solid instructions on how to do so. I haven’t gotten the DOM Inspector going yet, but see below for a method for getting the first two working in your XULRunner app.
First of all, I did all of this work using a custom built XULRunner on Linux from the XULRunner 1.8.0.1 source drop. I have not tried this on the stock binaries but I think it should work. In any case, here are the configure options I used:
--enable-application=xulrunner --enable-extensions=default,venkman,inspector --enable-svg --enable-storage
The Extension Developer Extension was pretty easy to hack into an XULRunner app. I just extracted the extensiondev.jar from the xpi and created an extensiondev.manifest that reflected what was in the install.js. I dropped both of these files into my app’s chrome directory and was then able to specify the FireFox overlay on my main window (chrome://extensiondev/content/firefoxOverlay.xul). It worked after creating a menupopup with the id “menu_ToolsPopup”, which follows the FireFox naming convention.
Venkman was trickier. I tried the same approach, ending up with a venkman.manifest and venkman.jar in my chrome directory. I then tried to overlay Venkman onto my main window (chrome://venkman/content/venkman-overlay.xul). The first hurdle is that the function toOpenWindowByType does not exist in the default XULRunner overlays. I just reimplemented it with the following:
function toOpenWindowByType(inType, uri)
{
window.open(uri, "_blank", "chrome,extrachrome,menubar,resizable,scrollbars,status,toolbar");
}
Then the problem became that it just didn’t work. I was using the venkman.jar that I extracted from the XULRunner build that I was using. I finally switched to the patched version of Venkman for Firefox 1.5 available from here. It worked like a charm. The main thing you have to do is uncheck the menu option Debug/Exclude Browser Files. Note that if you open more than one debugger window, the app crashes with a nasty segfault. My toOpenWindowByType stub should keep this from happening, but I haven’t gotten to adding any logic for that.
I packaged the glue for these things up into an xulrunnerdev.jar that also adds a menu item to open the JavaScript Console (in lieu of having to pass the -jsconsole command line). Here are the exact steps to get it working in your XULRunner app:
- Download the xulrunnerdev.zip file from below and extract it to your application chrome directory.
- Add a line to your application chrome.manifest for each of your application windows you would like to overlay with the tools. For example:
overlay chrome://testapp/content/startPage.xul chrome://xulrunnerdev/content/devOverlay.xul
- Add an appropriate menupopup to the windows you want overlayed. For example:
<toolbox flex="1">
<menubar>
<menu label="Dev Tools">
<menupopup id="menu_ToolsPopup">
</menupopup>
</menu>
</menubar>
</toolbox>
I have included a test application that includes all of this stuff below as well. There are certainly multiple ways to get this stuff to work. Hopefully this work can hold us over until more official support of the dev tools is included in XULRunner.
Downloads:
Building libgcj.dll for MinGW with GCC 4.1
August 19th, 2005Here’s some notes from my experience building libgcj.dll for MinGW using the GCC 4.1 (May 15, 2005 snapshot). This required a good deal of effort for me to figure out since quite a few things have diverged since the last reported sighting I could find of a working libgcj.dll.
Note that this entire process was done using a cross compiler hosted on Linux.
First, it should be noted that I upgraded libtool prior to building the DLL. When I get some time, I’ll rebuild it with pristine sources and make sure it works without this step. The basic process to upgrading libtool is:
- Copy libtool.m4 from a newer libtool distro to the root gcc source directory
- Run autoconf (version 2.59) followed by automake (version 1.9.3) in the subdirectories that will be built. Do not regenerate the config on the root directory unless if you make special arrangements to regenerate it with autoconf version 2.13.
- In the libjava target, you will manually have to patch the generated libtool after it gets created. You will need to change the “compiler_c_o” property in the GCJ section to “yes”. This is line 7278 of the version of libtool I used. Not doing this will cause problems when the Makefile tries to compile resources. There has got to be a better way to make this work, but I am not terribly familiar with libtool and didn’t look into it.
My primary motivation for upgrading libtool was to see if it would just automagically generate the dll for me. Sadly, this didn’t work but I think it tried. With that being the case, it is probably ok to skip the libtool upgrade altogether.
I also ran into a problem with the sequence of the build. During the first pass build, the gnu-java-beans.lo file is generated prior to the most of its classfiles being compiled. Because of this, it only contains a handful of classes, which causes a lot of undefined references when building the dll later. I think this is a general problem that just doesn’t manifest very often because the java.beans.* API is very rarely used in practice. A work-around is simple enough. After you build it once, just delete the generated gnu-java-beans.lo and rerun make. Since all of the class files are present the output file will be created as expected. You should see that the backing .o file(s) are much larger the second time around.
The next step is to create the actual dll. You should be aware that this is an extremely memory intensive operation. The memory usage of ld tops off at about 630MB on my machine. If you don’t have at least this much physical memory available to ld, your machine could thrash for hours.
I used the following script to create the dll:
#!/bin/sh
LIBGCJ_A=libgcj-static.a
mkdir libgcjobjs
cd libgcjobjs
ar x ../$LIBGCJ_A
cd ..
find libgcjobjs -name '*.o' > libgcjobjs.list
i686-pc-mingw32-ld --shared -Bdynamic -e _DllMainCRTStartup@12 -o libgcj.dll --out-implib libgcj.a /.1/home/tlaurenzo/gcc_cross/bin/../lib/gcc/i686-pc-mingw32/4.1.0/../../../../i686-pc-mingw32/lib/dllcrt2.o -L~/gcc_native/i686-pc-mingw32/lib -L/.1/home/tlaurenzo/gcc_cross/bin/../lib/gcc/i686-pc-mingw32/4.1.0 -L/.1/home/tlaurenzo/gcc_cross/bin/../lib/gcc -L/home/tlaurenzo/gcc_cross/lib/gcc/i686-pc-mingw32/4.1.0 -L/.1/home/tlaurenzo/gcc_cross/bin/../lib/gcc/i686-pc-mingw32/4.1.0/../../../../i686-pc-mingw32/lib -L/home/tlaurenzo/gcc_cross/lib/gcc/i686-pc-mingw32/4.1.0/../../../../i686-pc-mingw32/lib --export-all-symbols --enable-runtime-pseudo-reloc --allow-multiple-definition `cat libgcjobjs.list` -lmingw32 -lgcc -lmoldname -lmingwex -lmsvcrt -lm -lgdi32 -lws2_32 -lmingw32 -lgcc -lmoldname -lmingwex -lmsvcrt -luser32 -lkernel32 -ladvapi32 -lshell32 -lmingw32 -lgcc -lmoldname -lmingwex -lmsvcrt
(This script is just for demonstration purposes. You will obviously need to update all of the paths appropriately)
To build programs that link to the dll, you will need to use some extra switches:
i686-pc-mingw32-gcj --main=Test -L. -o test.exe -Wl,--enable-runtime-pseudo-reloc Test.java
If trying to duplicate this work, note that I am not working on pristine sources. In particular the changes described in the java@gcc.gnu.org mailing list thread entitled “Linker name conflicts due to optimization in gjavah” are critical for this to work.
Adventures with Java 5 and GCJ
August 15th, 2005A few days ago I set out to do something that had seemingly never been tried, yet the process seemed straight-forward enough and I anticipated few problems (famous last words, I know). You see, I am working on this client-server systems management application which requires a small agent to be installed on a number of client machines (Windows, Linux, MacOS, etc). The server side is written with Java 5. I entertained some thoughts of writing the agent in C++, but my C++ is a little rusty and I haven’t had much experience in managing a full-blown C/C++ cross-platform initiative… besides, I like working in Java. So, throwing caution to the wind, I set out to see if a natively compiled version of my Java agent would be possible and acceptable (I don’t want to contend with a huge JRE install or the memory footprint associated with it). Just to make life more interesting, my development environment is Windows XP.
So, here is the basic process I intended to follow:
- Install Mingw/Msys (I already use Cygwin heavily but want to be closer to the Win32 api and not have dependencies on cygwin1.dll)
- Use RetroWeaver to convert my Java 5 class files to Java 1.4 compatible equivilents.
- Use the RetroWeaver reference verifier to find references to classes/methods which are new to Java 5.
- Create a compatibility layer to call through to new features of Java 5 (note that from the outset I had been trying to avoid most new APIs so this step wasn’t so bad. Most of the stuff I found was as simple as using Arrays.toString(…) in test classes)
- Use GCJ to compile my classes and dependencies natively.
Well, I hit my first snag with RetroWeaver. While I was able to convert the class files, the reference verifier was buggy and difficult to invoke from ant. Poking around on the SourceForge site, I found a number of patches to fix the things I was having problems with. Unfortunately, the author hasn’t released an updated version of RetroWeaver in over 6 months and many of these patches conflict with each other or do not note properly what version of the source code the diff was generated against. Also, for some reason that was probably my own brain-damage, I was having a particularly bad time getting patch to actually do its thing. So, if anyone is looking for a version of RetroWeaver that has what I consider to be the right set of patches applied, you can get it here. This version seems to have a “mostly working” reference verifier, and the Ant task has been updated to make it easy to invoke, like so (assumes that the directory ${common.dir}/retroweaver contains all of the jars found in the RetroWeaver distribution. also assumes that retroverf.dir contains a copy of all class files, retrort.file points to a JRE 1.4 rt.jar file and std.classpath and depend.classpath are valid path constructs that contain all the needed dependencies for your classes):
<taskdef name="retroweaver" classname="com.rc.retroweaver.ant.RetroWeaverTask">
<classpath>
<fileset dir="${common.dir}/retroweaver">
<include name="*.jar"/>
</fileset>
</classpath>
</taskdef>
<retroweaver srcDir="${retroverf.dir}"
verify="true">
<classpath>
<pathelement location="${retrort.file}"/>
<path refid="std.classpath"/>
<path refid="depend.classpath"/>
<pathelement location="${retroverf.dir}"/>
</classpath>
</retroweaver>
Having spent far to long attempting to get RetroWeaver to do everything I wanted it to, I naively thought that GCJ would just work and life would be good. I couldn’t have been more mistaken! When I tried to compile my main jar file, I got an obscure error from the assembler complaining about duplicate symbols. I was on my way down the rabbit hole now! I recognized the duplicate symbol name as the mangled form of a method on one of my classes. I looked it up and saw that the method in question was an implementation of a method from an implemented generic interface. Looking at the byte-code revealed that the class had two methods with the same signature and differing return types. I vaguely recalled some discussion about covariant return types and bridge methods in Java 5 and the problem started to become a little less murky. The error spit out by the assembler included a mangled name that did not have the return type encoded in it, and that was apparently causing the name collision.
Doing some googling and sifting through the GCC bug database revealed that this bug is officially listed as bug #9861 and it has been open since 2003 (apparently the original bug was noticed while using one of the old prototype generics compilers). There was some comment that a new ABI was expected to fix this problem, but a look at recent snapshots of GCC revealed no fix yet. Well, it started to look like my original idea had hit a dead-end.
For some reason, though, I didn’t stop there. I downloaded the May 15, 2005 GCC 4.1 snapshot and started hacking. Now, the last time I actually tried to build GCC was back in 1998 on a Slackware Linux box. I don’t remember much except searing pain and an eventual partial success. To be fair, the GCC build has gotten a lot easier since then… at least if you are using Linux. If you are trying to build GCC on Windows using mingw, just stop. I eventually settled for the procedure outlined by Ranjit Mathew on his website for first building a cross-compiler on Linux, then using it to build a native compiler for Windows/Mingw. He includes some scripts that work really well for this purpose. Of course, there is a reason why you can’t find pre-built binaries for any GCC >= 4.0 on Windows/Mingw. There are two compilation problems that keep the suite from being built on mingw. The patches for these two problems are included in the main patch below. I’ll get them to the GCC team eventually, after making sure that the fix already isn’t in CVS head.
So I set my old Linux box to building the GCC compilers and went camping for the weekend. I just picked up the results today and started looking for actually how to fix the original problem. It seemed to me that the goal should be to modify the mangling routines so that they include the mangled return type in the name passed to the assmebler. For GCJ, this is actually really easy to comprehend and implement… just a one-liner change. The problem is, however, that with this change, the C++ and Java ABI’s no longer match up. This may not seem like a problem at first until you consider that all of the native parts of libjava are written in C++ using CNI, which requires the mangled names of methods match up on the C++ and Java sides of the house. It took me quite a bit longer to paw through the C++ compiler to figure out what to do. It would seem that including the return type in the mangled function name is already done for function templates (I think… I didn’t trace this down all the way), so the fix involved just adding another condition in which to include the return type. So, if the function is a method of a class and if the class is a Java class (descended from java::lang::Object), I do the same thing as is done for that special template case. There are some macros in G++ that make this a one-liner change as well… it’s just a much scarier change for the uninitiated because the G++ compiler is really complicated!
In conclusion, I think it is a Good Thing to have a working path for moving from Java 5 source code to native binaries and I hope that this article can help make that a reality (at least in the interim, since the “official” support for Java 5 in GCJ seems to be a way off). I also think it is really important to have an up-to-date GCJ compiler for Windows and am going to pursue making actual builds available. For now, though, the patch will have to be enough for anyone interested in duplicating my work. Here it is. I haven’t finished the clean build of the compiler and test suite yet, but have visually inspected and verified the results of this patch and it seems to be ok. YMMV. Since this is a breaking ABI change, I wouldn’t anticpate seeing it anytime soon in the official GCJ releases.
UPDATE 8/17/05:
It turns out that there was one additional thing that complicated the process of changing the mangling scheme. This was causing unsatisfied link errors when calling, either directly or indirectly, a number of static methods on the Math class. Since this was clearly a problem with the Java side of things, I breathed a sigh of relief (for not having to delve back into the C++ compiler). I did an objdump on the Math.o file and looked at the disassembly for the round(float) method, because it called the Math.floor(float) method which was one of the unresolved symbols. I was surprised to find that even though the Java code for round(float) calls floor, this call was nowhere to be found in the disassembly. At this point I realized that the compiler must have some mechanism for inlining some operations. So I did a grep for “floor” in the gcc/java directory and found the following:
builtins.c: double_ftype_double, “_ZN4java4lang4Math5floorEd”);
There was my unresolved symbol without the return type mangled into the name. There are 13 of these in the builtins.c file. Updating them with the proper signature makes everything work correctly.
File List:
- retroweaver.zip - 8/15/2005
- mingw-java5-gcc-4.1-20050515.patch - 8/17/2005 - Combined patch against the 05/15/2005 GCC 4.1 snapshot to enable name mangling and mingw support.