gtuhl: startup IT and software development

One aspect of a software startup that cannot be escaped is money must be spent on technology and development of technology. Whether this is a good or bad thing depends on if you ask the engineer or the accountant. My general rules of thumb are:

• Purchases that help people do their jobs better or faster are worth paying for.
• Before spending money on something look for an open source alternative that is cheap or free. Often you will find something better or only slightly inferior to the commercial item.
• If you are going to spend money on something, the price-to-substance ratio is important.

And now a smattering of thoughts and plugs for each rule of thumb in the context of our company that is full of my personal opinions. I do realize that the earliest days of a startup largely must ignore most of this list. For example, when you don’t have an office yet (and everybody works from their homes) you don’t really worry about getting comfortable chairs, good machines, etc. for that office.

Purchases that help people work

• Screen real estate is important. I used to think this meant 2 screens but have refined this to mean total resolution. With my macbook pro and spaces I went from using 2 computers and 3 monitors to just 1 laptop and I feel more efficient now. I like to give 2 monitors to any person that wants one - especially engineers, designers, and QA.
• Good chairs are worth paying for. I’ve worked places in the past that gave their engineers hand me down garage sale garbage to sit on. The nature of a software company means people are going to spend a lot of time sitting and the chairs need to be good enough that people don’t notice them all day (and often longer given the nature of startups). Aerons are great if you can get a deal on them but there are solid options in the $200 - $300. CWC sells better quality furniture at the best price.
• Don’t skimp on workstation hardware. I personally think the mac path is worth the premium for developers. On a per-item basis the price is virtually equivalent but given Dell’s willingness to haggle and price slash (especially if buying multiple items) a premium does remain. I think it is worth it.

Open Source

• We use Java and I think it is better than .NET and it is free. You can build it on Windows/Linux/Mac and you can deploy it to all 3 as well. I think PostgreSQL is better than SQL Server (and MySQL). The Microsoft lock in has never made any sense to me and I feel the Java community is a great place in that the number of unqualified engineers is relatively small and it is full of extremely qualified people. Java also scales vertically or horizontally very, very well. It has the whole 10,000 frameworks/libraries to choose from “problem” that .NET does not have but that is okay in my opinion. We went with Spring/Hibernate/DWR and it has worked out great.
• PostgreSQL is fantastic. The developers are accessible and helpful and the community is strong. We’ve run it up to a 1TB database and it handles it just fine. You obviously have to run it on a reasonable machine as load increases but it scales vertically wonderfully and there are addons for replication. Check out Slony and/or Mammoth Replicator if you need that replication, we haven’t yet. Visit this site for installing Postgres on your local mac workstation.
• Linux is the way to go for servers. I don’t think the Linux/Dell combo can be beaten on the server side.

Price-to-Substance Ratio - Some Examples

• IntelliJ IDEA is worth its cost. It is magical and exceeds a plugin-ridden eclipse install for features out of the box and I think the editing experience and source control interaction are superior.
• Despite stability issues I think the Leopard incremental upgrade to OSX was worth it for productivity overall. Spotlight and Spaces have changed my workflow completely.
• Dell provides a fantastic ratio here. I would strongly recommend them for server hardware, especially their latest models. Solid architecture, solid raid controllers, RAM, etc. If you go with Dell get in sync with a Small Business team. It will save you money and streamline the process as you get to talk to the same people every time. Their business lines of laptop (Latitude) and desktops (Optiplex) are also solid.
• Good consultants and contractors are worth their rates for focused, time-constrained assistance. You have to be careful though because there are a large number of unqualified people posing as consultants and contractors that aren’t worth the time it takes to arrange a contract. If you find somebody you can work with and does a good job keep using them as needed.
• Parallels is worth its very manageable price for providing IE6/IE7 testing to mac-using developers. See this post for help setting up the free VMs provided by Microsoft for doing this testing.
• FlexBuilder isn’t worth the cost. When I used it a long while back it was $700+ with charting and had marginally more functionality than notepad2. Following that link, it looks like they are pumping Flex 3 now. The fact that Flex 2 has profound issues makes this especially troublesome.
• Flex Data Services pricing defies all reasoning. $20k per CPU. Same for pretty much any other product that charges per-CPU. If anyone knows of ANY per-CPU product that is worth paying for let me know. I recently priced out a better WYSIWYG editor for portions of our product and they wanted pricing per CPU for a text editor.
• And finally, I think sharp, qualfied engineers that you can interact with in person in the US are superior to any offshore team. When you consider the time differences, communication barriers, and general lack of quality offshore I believe a 5 man team of people that know what they are doing and work together here could out perform a 50 man team of offshore cube farm drones. I have 3 specific experiences (admittedly not that many) working with offshore teams. 2 ended in utter failure to complete the task, and 1 was bailed out of before it got too far along because even the onshore PM/BA assigned were completely clueless and ineffective. I feel like the offshoring development companies live in an alternative universe where you just keep a neutral look on your face through meetings and shuffle out inferior product making fixes until the customer is too frustrated, tired, or so accustomed to the low quality that they start to believe the software is good and consider the project a “success.”

So there you have a smattering of my thoughts. I expect to elaborate on many of these items in separate posts in the future. You can likely tell by the tones which items I find most interesting and/or alarming.

I’ve talked about it before, but we’ve got a chunk of code in our primary product that allows permissioned users to build extremely complex queries using a GUI. The GUI inputs are converted to an intermediate format AST that is used for persistence and this gets converted into the Hibernate Criteria API equivalent by the DAO when the built up query is executed. The resulting feature is pretty dang powerful and I actually think it would be a cool thing to wrap up into a clean package and provide for others whenever I have time to do so.

In any case, this thing is complex enough that I’ve encountered many bugs or edge cases that aren’t handled cleanly in the Criteria API itself and we found a new one (new for us, the JIRA issue has been around for awhile) the other day that is worth throwing up a post about for others to find the workaround for more easily.

Sometimes databases require some denormalization for performance reasons (joins between 8+ digit row count tables don’t care how much RAM you have). If the column count gets too intense though and if you only really need a few values of each record the IO required to pull out all of the data can be fairly expensive and projections can be needed. With the Criteria API you can project specific columns with something like this:

public List<Object[]> getNamesOfPeople(){
    Criteria query = getSession().createCriteria(Person.class);

    ProjectionList projections = Projections.projectionList();
    projections.add(Projections.property(”firstName”));
    projections.add(Projections.property(”lastName”));

    query.setProjection(projections);

    return query.list();
}

One downside to this approach is you have to deal with Object arrays as the result from your queries. If you have manager or controller code its a bummer to have to write chunks of code for marshaling the values in the Object arrays into the actual objects (Person in this case) before doing things with it. Queries built with the Criteria API can have result transformers applied to them. Without going into specifics i’ll just say there is an AliasToBeanResultTransformer that can be used to automatically dump projected values into a instance of the specified class. Now the code would look like this:

public List<Person> getNamesOfPeople(){
    Criteria query = getSession().
        createCriteria(Person.class);

    ProjectionList projections = Projections.projectionList();
    projections.add(Projections.property("firstName"), "firstName");
    projections.add(Projections.property("lastName"), "firstName");

    query.setProjection(projections);
    query.setResultTransformer(
        new AliasToBeanResultTransformer(Person.class));

    return query.list();
}

The way the AliasToBeanResultTransformer works is to check the names of the properties of the object you want and write into them any values from the result with the same name. In the above example the second parameter to ProjectionList.add is the alias or the label for the column when the result hits the AliasToBeanResultTransformer. The above code will yield a list of Person objects with their firstName and lastName properties set. If the alias parameter to ProjectionList.add is omitted you will get a list of Person objects with nothing set (because Hibernate will use its own aliases and they won’t match up).

Finally the bug. When Hibernate generates the actual SQL that hits the database it aliases each projected column with its own internal scheme for the purposes of the SQL query. It will generate SQL that looks like this:

select this_.first_name as y0_, this_.last_name as y1_
from people this_;

If you add restrictions to your Criteria API query that involve the same properties and aliases you used in your projection list you can end up with generated SQL that looks like this:

select this_.first_name as y0_, this_.last_name as y1_
from people this_
where this_.y0_ = 'John' and this_.y1_ = 'Smith';

The key thing to note is that Hibernate is trying to be smart and reuse the alias specified in the select clause. The problem is that this isn’t valid SQL to any rdbms I know of. If you try to use the alias like that in the where clause you will get a “column ‘y0_’ does not exist” in PostgreSQL and similar messages in other databases.

This is a long known issue and the corresponding JIRA issue can be found here. Basically someone reports it, he gets cut off and told its Oracle’s problem because the SQL looks fine, and people pitch in about pretty much every other rdbms also not supporting it. Its still marked Minor and is Unassigned after 2 years so I doubt it will be fixed anytime soon. Thankfully, if you are having this problem you can look deeper in the comments and get a CustomPropertyAliasProjection class that when combined with a modification a little further down can be used to make it work with the AliasToBeanResultTransformer. The fix is essentially to ignore the alias in the select clause when generating the where clause. I won’t repost the code here as I didn’t write it and the JIRA page works just fine.

So if you run into this problem that is the fix. There are actually lots of bugs in the Criteria API but until someone is able to build an ORM tool better than Hibernate (and no such thing exists to my knowledge) I’m not going to switch and the fact that it is open source means you can usually find a workaround online.

In our main product we’ve built a pretty heavy-weight, powerful block of code for allowing users to build and execute very complex queries through the GUI. The permutations covered by this widget recently uncovered a hibernate bug and want to quickly write up this overview so google can start indexing it

As a short explanation, the queries being generated by the code backing this GUI-based query builder was generating fragments that involved type casts. We are using PostgreSQL and they looked like this:

(some_column > 0)::integer

This has worked great for awhile but last night I uncovered a combination of factors that causes this to break. If you are:

• Using the Criteria API to build up a query
• Using sqlRestrictions in your query
• Using colons in your sql restriction (in my case it was the ::type casting for PostgreSQL)
• Have a filter applied to your queries. In our case it was a filter being set by a @FilterDef annotation on the model object

If you combine these factors you will get an error along the lines of Invalid filter-parameter name format. When I googled this I ran into the common scenario of only finding hits within public-facing code repositories for the project throwing the error. Hopefully the next searcher finds it here.

Basically I think what happens in this case is colons aren’t properly escaped and hibernate then expects named parameters. In my case it was seeing ::integer and thinking I had a named parameter called :integer and the code in SessionImpl.java of the hibernate project was blowing up because this doesn’t look like a valid parameter name. It would have been much harder to track down the problem if hibernate wasn’t open source.

If I disabled the filter it worked fine. In either case the complete workaround is thanks to PostgreSQL and involves using this alternate casting syntax:

CAST ((some_column > 0) AS integer)

Thank goodness PostgreSQL provides different types of syntax for this. I’ll probably post this to the Hibernate JIRA before too long, i’ll likely somehow be told it isn’t a bug.