armtuk: Cheetah (Default)
From the PHP homepage:

PHP 5.2.7 has been removed from distribution

Due to a security bug found in the PHP 5.2.7 release, it has been removed from distribution. The bug affects configurations where magic_quotes_gpc is enabled, because it remains off even when set to on. In the meantime, use PHP 5.2.6 until PHP 5.2.8 is later released.

This means that even if you used magic_quotes_gpc, which is evil anyway, but necessary for MySQL (reasons not to use MySQL and PHP), you would have been open to an injection attack, and that values that had single quotes in them like O'Reilly would have broken your site.

Can anyone say regression tests?

PHP - Not enterprise software.
armtuk: Cheetah (Default)
If I read this post on the MySQL Forum correctly - it appears that MySQL Doesn't roll back any transactions that were started by a connection that terminated prematurely. This means that any program that exists in the middle of a transaction can leave locks on the database open, and prevent other queries from running. Has MySQL not heard of MVCC?

MySQL News

Dec. 18th, 2008 12:56 am
armtuk: Cheetah (Default)
So in MySQL it's possible to have a table:

create table fish (
created_date datetime not null

and do

insert into fish values ('0000-00-00');

This is valid.

wow. Wow. WOW.

Let's face it - it's just not a serious product.

Java will of course fail to convert '0000-00-00' into a valid java.sql.Timestamp

I am going to have to write code that runs on initialization of the context that fixes the data in the database because MySQL is such a pile of rubbish.
armtuk: Cheetah (Default)
Today - I wrestled a database driven form out in just 50 minutes. That included coding the HTML for the form, coding the Data Object in Hibernate Annotations, creating the DAO, creating the processing ActionBean with validation, creating the display back HTML to display the data back out again, updating the Spring Context with the new DAO, adding the Database object to the hibernate configuration. Running the update scripts to change the database and testing the form to reveal that I hadn't done a couple of the steps above initialy.

50 minutes. We used to budget this kind of page at 4 hours. Stripes is awesome. Hibernate could generate more on the fly I still feel, but it's not bad considering I'm driving the DDL statements from the Java class. You have to write that code one place, either in HBM XML files, Annotated Java code, or SQL statements. I still think you could build the form HTML based on the database schema too which would save time. Maybe I'll make a helper application to do that.
armtuk: Cheetah (Default)
super computer science geekness here )
armtuk: Cheetah (Default)
So for today's hard knocks hibernate lesson we learn that putting annotations on Member variables instead of Member Functions is a really bad idea. It breaks when you use inheritance. If you inherit from a MappedSuperclass and that superclass has members that are complex objects, then when you inherit them, all the annotations are ignored in the subclass, so you end up with database fields being of type bytea (at least in PostgreSQL) instead of being correctly mapped as foreign keys.

So in summary: Don't put annotations on member variables, always put them on member functions.

*SIGH* time to refactor 32 database classes.

At least I found the problem quickly.


Oct. 18th, 2008 10:22 am
armtuk: Cheetah (Default)
Can I just say that IntelliJ rocks. It detected that the graphic in my background-image inline style in an HTML tag in my JSP did not exist, and highlighted it as a problem. How awesome is that?

Plus, I have killer head-ache of doom :( so I'm a bit cranky.


Oct. 16th, 2008 09:28 pm
armtuk: Cheetah (Default)
MVC means Model, View Controller. Not Controller who makes a Model and passes it to the View. This is the normal usage of this system in many implementations, but that's not really what MVC is is it? The Model is an independent piece that isn't tied to the Controller directly any more than it's tied to the view. The Controller is responsible for updating the Model, and the view is responsible for rendering the Model.

In a normal Web based application, the Model is in the Database which is often an RDBMS. It doesn't break MVC to have the view talk to the database, because the database is the Model. The big mistake of many ORMs is trying to represent the Model in two places, in the database and in the application server. This of course creates all the same problems as multiple master replication causes for a database product. Every transaction has to be replicated across both systems incurring a huge overhead and making transactions way longer than on a single master system leading to huge performance problems. It's pretty common sense that the best form of distributing load is to have a share nothing cluster where you put different parts of the data set on different nodes. This is in opposition to multiple master replication which puts all the same data in two places. This is good for redundancy, but bad for performance, unless your interconnect is as fast as your local storage, which is rarely the case (don't just think throughput, think latency too). Of course you can do the best of both worlds and replicate and distribute like RAID 10, but this means having homogeneous copies, not heterogeneous copies like relational data and objects.

If we think of MVC as a three legged stool as in the first, we have true MVC, but if we corrupt it, then we get Bad MVC like the second which is like trying to have a two legged stool. It doesn't work too well:

Presenting the Model is the view's job. Preparing the data for presentation is not the controller's job. The controller's job is to process user input and update the model. The model is not a transient object that's instantiated and reads data from a store to a Model object, that's a proxy. The real model is the datastore. Most RDBMSes I know have pretty good methods to update the data within them. Most Models need to do internal checking to ensure that they are sane too and aren't corrupt. This behaviour does not belong in the controller. It's part of the Model. That means database constraints and triggers.

Any framework that forces you to build the Model in the Controller and pass it to the view is a corruption of MVC and is asking for serious concurrency and performance problems. This doesn't mean however that your View cannot be started by a servlet or similar that reads data from the database into a hash and then forwards the display to a JSP or something similar. Just that to do MVC right, your controller doesn't return a Model and a View like in SpringMVC, it might decide which view to send the user to, which will then in turn access the model.

This means that we need different functionality in the view than most frameworks give I think. Some of this can be done in JSTL, but it's kind of clunky.

I'm going to read the rest of the Wicket book in the hope they get this, but I'm doubtful.


Oct. 16th, 2008 12:15 pm
armtuk: Cheetah (Default)
bzr-gtk for windows!! YAY! I'm happy about this. I can finally look at my version history without booting a linux VM.


Oct. 14th, 2008 07:01 pm
armtuk: Cheetah (Default)
Nope - it's MVC like everything else. Sad.


Oct. 14th, 2008 04:25 pm
armtuk: Cheetah (Default)
I have just started looking through the book on Wicket. They seem to have the same feeling towards MVC that I do: it's a fundamentally broken approach to web sites. I'm glad I'm not insane :). I got a chance to read some of it whilst Thomas got his hair cut. I am hopeful that the rest is as good as the start.
armtuk: Cheetah (Default)
I just discovered that I have been committing my changes to the mls project to the wrong branch. Ugh!! But git to the rescue! I can switch to the correct branch, and cherry pick the correct revisions onto the master branch. Ha-ha! I wield my git at you stupidity! All better now, and it didn't even take that long to fix.
armtuk: Cheetah (Default)
So stracing postgres shows that my function deg2rad was segfaulting. Wow - apparently some query is calling deg2rad and passing in a null value. The function doesn't like that, so I've put in a test to make sure if it gets a null, it returns a null. So far it's looking promising. I could now submit the admin page that I was using before to test the problem, so my reproducable case has gone away. The other application that was doing it lots was trac. I'm not sure how trac's crashes can be attributed to my stored procedure in an unrelated database, but a bit of clicking around in trac has shown no problems yet. Tomorrow will be the big test. It seems to happen during the day when the server is under load much more prevalently.
armtuk: Cheetah (Default)
We are having a major malfunction with our database server that started about three days ago. For some reason database connections are closing unexpectedly or at least reaching EOF when they shouldn't be.

I did all the normal stuff. Downgraded Postgresql from 8.3 back to 8.2 wondering if that was it. That didn't fix the problem. I rebooted the server, that didn't help. Restarted everything like ten times. I posted to the PgSQL mailing list, and I posted a ticket with Rackspace. The PgSQL mailing was as always fairly helpful, but Rackspace was worthless. If there was anybody else who even answered customer support requests on a reliable basis and gave bigger servers, I would totally ditch Rackspace, but they are the best of everyone we've worked with which is sad. Not much of a recipient of fanatical support so far (ServerBeach has been excellent, but they don't really provide the kind of server we need).

After few responses that were helpful in fixing the problem I decided that the best next step was to strace tomcat to figure out what was happening on the socket. The first strace the socket threw a SIGPIPE and tomcat's thread Segfaulted :( (but didn't throw a Java exception so it looked like nothing happened from the log file). Unfortunately I wasn't expecting tomcat to crash (we've seen the DB process segfault before, but not tomcat) so I wasn't running with a non zero core dump limit.

The next runs revealed little of anything. Then I got one that had more clear information in it. It looks like tomcat is sending a database request to postgres and the recv comes back with a 0 return value, which means postgresql is closing the connection. Super wierd.

I've posted this info to the Postgres mailing list. Not sure what else I can do at this point. Maybe my postgresql.conf is stupid, but I haven't changed it recently I don't think, so I don't know why this started happening all of a sudden.

I think my next step will be to strace postgresql, but that means setting up a new instance because I can't really strace the production instance of postgresql as that would generate so much log, it would be impossible to go through it all.


Mar. 4th, 2008 10:57 am
armtuk: Cheetah (Default)
I have long spurned the use of IDEs preferring rather to rely on trusty tools that work on the command line that have been so well debugged that they just work. Vim, ant, sed, awk were my best friends in coding land. Then I had to develop a desktop application that had to run on Windows. In Windows Vista, cygwin and java do not get along, and cygwin doesn't work very well at all in Vista particularly. I turned my attention therefore to the leading IDE available for java: Eclipse (I poked NetBeans, but it sucked compared to Eclipse, in particular it couldn't import my ant build scripts successfully, and doesn't deal with other formats well). I started to work with it, and found that it's fairly easy to work with, and things are well set up and easy to find for the most part. Then I discovered it could handle my python files in the project (The project has a bin directory that has various python scripts that do trivial database operations that are too big to want to do in a stored procedure that gets smooshed into one transaction). Then I discovered AspectJ. I cannot imagine trying to work with AspectJ outside of an IDE that can figure out where your pointcuts are applied dynamically. It would be a total nightmare, and I'm getting very fond of AspectJ features, to the point that I purchased the AspectJ book. Then there is web.xml editing. Frequently when I edit this important file by hand, I make simple typing errors which causes a restart of tomcat over and over again till I get the format corrected. This does not happen in the IDE as it's presented as name value pairs where the name of the tag is pulled from the DTD and presented in a drop down menu when you want to add a new tag. This is getting particularly important as I am starting to shun the use of JSTL in JSP for submit pages, and move back to using pure servlets for this functionality after a big blow up with a database upgrade. All these things have dramatically reduced the time I spend fixing trivial typos in my project and allowed me to be more productive. I wrote a PDF library that will be used on multiple projects in about 4 days straight including tests with some Aspect Oriented features. I don't think that would have been possible on the command line.

My one and only complaint is that there is poor git integration. There is an egit project that is a pure Java client for git, but it doesn't work very well, and so I have to go back to the command line to work with git. This is a general weakness of git though. They have the wonderful gitk program, but when it comes to visual interfaces, particularly on Windows, it is severely lacking. Given some of the advanced features of git that I have had occasion to use like git filter-branch, I can't imagine switching to another VCS unless it had those features, which I'm pretty sure no other VCS does (though I'm looking into bazaaar).

Another nice feature is Eclipse's ability to capture stdout and stderr from various sub-processes like tomcat, and JUnit testing which allows me to see right in my IDE what's going on with the server or with my test cases.

In short, it appears that I am eclipse's newest convert, though to be honest, to make the best use of Eclipse you need a monster monitor for all the views, but I have that mostly taken care of with (New Egg 22" Acer wide-screen for just $229!).

There are many other features of eclipse that I am going to have to explore. It has some very nice looking plug-ins for creating UML documentation, and documentation is something we are lacking a bit. I tend to do a good job of keeping things in my head, and as there isn't another developer, I am a bit guilty of not documenting what I have built very well (though the code is commented, but I hate trolling through source code personally). Generally most of what I build is too simple to warrant complex UML diagrams anyway, they are mostly just websites that process basic information to a database in simple submit processing type forms, and basic search functionality on top of that data.
armtuk: Cheetah (Default)
I am writing some PDF functionality as you may well know if you've been reading this feed lately.  And I have come to the part where I write out a PDF object and it's children to a file.  Silly me, I figure I can use object inheritance to solve my problem:

public class PDFWriter {
    public void write(PDF p) {
       PDFObject a;
       for a in p {

    public void write(PDFName a) {

    public void write(PDFDictionary a) {

of course what I am trying to do here <i>isn't</i> a feature of object inheritance, but rather a piece of functionality referred to as double dispatch.  And tonight I found out that Java doesn't <u>do</u> double dispatch.   Bummer said Dougal, time to figure out something else.  And for the first time introducing a design pattern that isn't obvious, the Visitor pattern.  Ok - so the essence of the pattern is that we use polymorphism/inheritance to act like double dispatch by calling a method on <pre>a</pre> instead of calling a method on the Writer passing a generic object.  The method on a is of course called on the inherited type rather than the base type, and it calls visit(this) on the visitor, which is the PDFWriter object.  All that remains to do is rename the methods called 'write' to 'visit'.  Pretty nifty.

That's my lesson in OO for tonight.
armtuk: Cheetah (Default)
Over the last couple of days, in an effort to simplify the development of the PDF Library I am working on, I have been fiddling a bit with aspect oriented programming. I have been hearing increasing amounts about aspect oriented programming for some time, so I finally decided to look into it a bit more. I found a PowerPoint presentation that was given by the Xerox Parc people about AspectJ, the Aspect Oriented wrapper for Java, and read the first 2/3 of it, and started experimenting. This is where I found that the PP was way old, and was in-fact using an archaic syntax that was no longer supported. I downloaded the plug-in for Eclipse that supported AspectJ and that came with some examples that cleared things up. Yesterday we went to Borders Books in King of Prussia, and they had the AspectJ in Action book, so I picked that up and have started perusing the contents thereof, although with an eye to the functional as I am working on stuff that really really needs to get done in a hurry. I have used a couple of aspects now to really help debugging the work I am doing, and it's very cool. Really cuts down on the insertion of System.out.println() calls in the code, and keeps the code nice and clean. So far I really like it.
armtuk: Cheetah (Default)
I am working on a new application for Pennmark publishing. This is a neat little application that sources images from a client and uploads them to a server for processing. I have decided that in this wonderful internet age, the application should be able to report any errors back to the server, so I am building an infrastructure to do remote error reporting. I really think this will be a big help to us when we are trying to debug what is going wrong with a clients instance of the system. I think I am also going to build an auto-updater too that will pull down the latest jar file of the software when it starts up. I have forgotten how much I love application programming. This is the stuff I really enjoy best of all. It reminds me of my days at Octagon building a PDF viewer in pure Java. That was an awesome project too. I might have to re-use some of that knowledge in this project to be able to do some PDF shinanegins on the server processing side. That should be a blast too. Today for the first time in awhile, I really do feel like I am riding the world.
armtuk: Cheetah (Default)
After the last time I got paid, I decided that it was about time to teach myself more about design. Sarah always said that I could learn it if I wanted to, and I do. I purchased for myself two books that came well reviewed on "Layout Workbook" by Kristin Cullen and "Color Design" by AdamsMorioka and Terry Stone.

I have made it about half way through the colour book, and it's very eye opening. It takes you through some basic colour theory and talks about colour combinations and what concepts and meanings are associated with what colours in western culture, and then also talks a bit about some colour associations in other cultures, particularly Japanese and Chinese. It goes through various concepts as to how to contrast colours using various techniques and how to pick colour groups. It gives some concrete examples of colour groups that work well, and takes you through a set of example designs and their chosen colour schemes.

The second book on layout is all about the general approach to graphic design, and contains various concrete steps to producing a good design. It talks about each element of the design in turn and takes you through how to be effective at it. This book ultimately says that nothing can replace intuition and creativity which above all else help you make a design. I think I am going to have to read this one at least twice. It's a bit light on details though which is a bit frustrating to me being a technical and detailed sort of person. I like to know the mechanics behind everything, and I inherently want to figure out the mechanics behind good design. Perhaps there are only guidelines, and I have to figure out the rest myself. How counter-me is that?
armtuk: Cheetah (Default)
So I continue to work with Plone. This week I have been reading up about object inheritance in zope and how properties and methods are acquired through the object tree. Highly confusing, but it reminds me a great deal of the MUD I wrote a number of years ago. I haven't quite figured out how the two inheritance paths work yet, although I get that one of them is the inheritance path through the container and it's container etc. This makes for a very interesting system that will acquire methods from parents and do the default unless you make something more specific. Very easy to override and work with. I am excited about this.

On a slightly more sour note, I have observed that buildout is broken. The ZODB scripts in the buildout try to execute with python2.3 which is not installed as compared with the other scripts that execute with python2.4. All the zope db management scripts are broken. What's worse is that I posted to the lists, and nobody gives a damn. This is so typical for FOSS. Programmers that don't care about the little things that make or break a product. I would offer to help fix it, but somehow I doubt somebody will take the time to take me through the process enough so that I can fix it.

The next chapter of the book starts us actually building some custom content types and fleshing out the functionality that goes with them. I am very interested to work through this as I am contemplating using custom content types to represent agents and offices within our real estate system. Maybe even properties, and migrate away from JSPs and Tomcat. I always liked python better than java, it just suffers from being slow. Although having said that most of the slowness in our current system is from the database calls to build the darn drill downs. It's cool functionality, but it really loads the system up pretty good. There is also a bug with this functionality. The drill downs are supposed to save themselves so they don't have to be rebuilt with subsequent page loads, but it looks like that's not working, and they are rebuilding with each page load. Not a high priority problem, but I should get around to fixing it sometime.


armtuk: Cheetah (Default)

April 2017

16171819 202122


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 20th, 2017 12:50 pm
Powered by Dreamwidth Studios