I am neither architect nor engineer but an ecologist and gardener

This discussion has been going on elsewhere, and it resonates well with me. It is useful to remember that all four terms are analogies. The thought space about my job as being a software architect or engineer are quite old, the ecologist and gardener ideas are refreshing.

Applicability of "Google’s three rules"

Robin Harris wrote about "Google's three rules" for the data centre, in short they are:

1. Be cheap
2. Embrace failure
3. Architect for scale.

Great rules and they appear to be working well for Google. Does that mean that your organization should adopt the same rules? Not necessarily. As you read through Harris' discussion a couple of key success factors jump out at me that I suggest are pre-requisites of the Google approach.

1. "Free or home-made software": If you are paying license or support fees for the software you are running that has a huge impact on how you approach provisioning. For many Enterprise Data Centres (EDCs) I would expect that they have already discovered that the cost of the software stack can easily exceed the cost of cheap servers several times. That is very much approaching Gates' "hardware will be free" prediction. Schwarz said a similar thing as well. What remains that drives cost is the software stack, and if you want to put large multipliers in front of that cost, then it has better be a small number.
2. "Google hired some of the best minds in the business": rather than investing in software licenses or expensive hardware, they are investing in brain power. It seems to be a reasonable approach to me? Have they figured out how to scale brain power? Certainly there are lots of untapped talent around the world that should satisfy demand for a while - how long depends on how many companies take the same approach. The more significant limiter is not supply but rather their ability to manage the bureaucracy that will evolve. But this is not about Google, but your organization. Can you acquire brain power and scale it?

I am not saying the Google's approach is wrong - but if you don't satisfy those pre-reqs and forge forward with their approach, you will have problems. Lots little and yet expensive problems called servers.

Event Processing

I've come upon lots of good reading lately on complex event processing. Interesting stuff indeed. Although there are some really cool things that can be done with this kind of technology, I think operational costsare a significant stumbling block for organizations like mine.

The article "Event Processing in 2007 and beyond" provides a good overview and some examples.

This site is dedicated to the topic and contains much good material.

Design patterns is a favourite topic of mine, and CEP Design Patterns provides a design pattern spin.

Multi-processor and Programming Languages

"now that multi-core is in rapid adoption, the issue of multi-processor programming is beginning to loom large." I wasn't the first to say it, but it is reasuring to see others say it to.

With respect to StreamIt, it is a completely different programming model than the previously reported Fortress Language.

A legitimate alternative to passwords?

| Tech Sanity Check: has a commemtary about yet another Multi-factor Authentication scheme.

vidoop has this interesting scheme for multi-factor. The interesting twist in this case is that the scheme has the potential to be based more on how you think than a known fact. You might select a sequence of pictures containing boats, airplanes, and cars. The theme could be transportation, the colour blue or aluminium.

They aren't there yet - it seems their current scheme relies on you thinking they way they do. But it does strike me as an improvement over static images.

Shared Items

If this works out right, you should now see a shared items feed near the top of this page. What are shared items? Well many sites produce a summary of their content in a form that can easily be syndicated - that is consumed by other sites, or applications. Often referred to as a 'feed' or an 'RSS feed'. RSS is a particular protocol used to implement this feature. ATOM is another such protocol, although they often get lumped together and called RSS.

So what can you do with it? There are many ways by which you can consume these feeds. Internet Explorer 7 has such a feature built in. So does my.yahoo. My favourite mechanism is Google Reader.

If you know somebody who reads a lot of internet content, you may wish you could 'read over their shoulder' every time they say 'Hmm. That's interesting!' Well that is exactly what Google reader shared items let us do. If I find something if interest, I can mark it 'shared', and it will then show up on my shared items feed. If you'd rather just look at a web page of the same items, you can do that instead.

There is one thing I would like to be able to do, that 'shared items' doesn't allow. And that is to make a comment on an item I share. Well maybe we don't need that. That's what this blog is for!

Have great day.

Service Architecture - SOA: Five Nines in a Service Oriented World - Part 1 the problem

Service Architecture - SOA: Five Nines in a Service Oriented World - Part 1 the problem: "Five Nines in a Service Oriented World"
A nice little discussion about the math behind availability.

Picking the Open Source Winners

This is a big topic for me at the moment. I am very interested in the OpenBRR and Navica scoring methods and the work undertaked by ohloh and optaros.

Authentication, Authorization, and Context

I was reading James McGovern's blog today and it reminded me of a conversation I had at work yesterday. James focus is on vendor product - something that is certainly of interest to us, but beyond that we also have to deal with our internal applications.

There are three issues that always pop-up when we try to integrate a new software product.
How do we authenticate?
How do we authorize?
What was the user try to doin the first place?

We are starting to get a good handle on #1 - but I would like to see us authenticate at fewer points and establish a trust network between applications. SPNEGO, SAML, WS-Federation, Liberty, and perhaps OpenID are all promising.

The James' blog speaks to the second part - authorization - The next big frontier. What roles can the principal hold (for this application)? The application part of that sentence is ways controversial. This is where XACML fits in.

The third piece - what were we doing is a tongue-in-check reminder that the user was actually trying to do a job before security "got in the way". The user likely had some kind of established context that should, ideally, be available to the next application. Although not always the case, it is a frequent requirement. For example, the user may have been working with a customer in the CRM application, and now needs to work on the customers Loan in the credit management application. This customer and loan context information needs to be carried forward. There are no good soutions for this that I know of. This is the undiscovered country.

Any thoughts out there?

Will we need a Fortress for our cores?

Two news theme over the last several months strikes me as mutally relevant.

First there is the information coming from Intel about future core densities. The Gulftown processor will likely make its debut in 2010 and will contain 32 cores. Intel research is also working on an 80-core research prototype. As the InformationWeek article discusses, software will need to change to accomodate.

Speaking of software, Java is a significant development language for the business world. What will business applications do with an 80-core engine? Multi-threaded applications - wherein the threads are explicit to the programmer are not the way to go. To the typically programmer I say "You can't handle the threads".

So what is the second news theme? Fortress - a research effort coming out of Sun Research. It is targeted at High Performance Computing; A replacement for Fortran. Am I suggesting that we rewrite all of our applications in Fortress? No. But Sun is implementing Fortress on the Java Virtual Machine. So there are some good possibilities that future Java language features will be able to drive out the same multi-threaded behaviour for which Fortress is striving.

Bank loses data

Bloomberg.com: Canada: "CIBC's Talvest Mutual Fund Loses Client Data Files"! This cannot happen too many more times before companies will be forced to put the technology in place to ensure that data that leaves their premises is encrypted. That won't be cheap, but it seems that the cost is inevitable.

Historical revisionism

Historical revisionism - what does this have to do with technology. Well last month I was trying to figure out how to make Sametime display my picture. I found a very informative article about exactly that process. I thank the author for publishing the information.

However I was shocked by how difficult the process is. Perhaps suitable for the corporate world, but still awfully painful - especially when you compare that process to the drag-and-drop simplicity of MSN Messenger.

I made a civil comment on the topic - it seemed that the author and his readers may very well be infleuential in improving the process. I thought I was being helpful. I kept checking back to see if anybody responded to my observation. Sure enough. Today I discovered that my comment was deleted!

Now this is not a great travesty - certainly not in the same league as what the wikipedia artcile covers. And not the first time it has been reported in the online world. New York times artciles and Whitehouse pages have suffered the same fate. But it is the first time it has happenedto me.

But this is the internet, and this blog is the solution. Now you all know that changing your display picture in sametime is very difficult - and I won't delete your comment - unless it has something to do with Russian brides ;-)

The future of the ESB

Loek Bakker's weblog: Understanding the future of the ESB: "It's so simple: ultimately, when all the parts have the ability to work together (i.e.: they all can communicate through messages, which will be the bus), the role of the ESB will be like the role of DNS for the Internet: addressing and routing. Nothing more, nothing less."

I suggest that he overstates the role of DNS. It doesn't really do routing. That is the job of routers and switches.

I generally agree that the capability we want for the end-points to do the processing, so that the actually communicaiton flow is point-to-point. But that point-to-point connectivity is not exposed to the application. I want to be able to modify application behaviours and policies as if it all went through a central hub, but don't want to actaully make a network hop and who-knows-how-many context switches in order to perform any routings and transformations.

Using DNS as an example, browsers look-up (resolve) domain names only occasionally, and then direct their traffic to the end-point directly. In some cases the end-point may do some virtualization and wrokload management at the far end, but all of that is of no consequence to the requestor. This is the way the applications should work with an ESB.

Tim Bray On PHP

"Spaghetti SQL wrapped in spaghetti PHP wrapped in spaghetti HTML" Ouch. Tim certainly has an opinion here. I question where this has anything to do with PHP. One could certainly create the same kind of pasta using JSP and is the way many ASP sites look inside. I suggest the difference is not the language, but the focus of the developer.

Dan Bricklin's wikiCalc

Dan was the original author of VisiCalc. He is at it again with wikicalc. An online spreadsheet. I like the idea, the alpha code is somewhat functional, but needs polish. Great work Dan.
Software Garden Products: wikiCalc Program: "The wikiCalc program is a web authoring tool for pages that include data that is more than just unformatted prose. It combines some of the ease of authoring and multi-person editing of a wiki with the familiar visual formatting and data organizing metaphor of a spreadsheet. "

Designing Software for Your Market

One thing that bothers me is the way that many IT shops go about desginin infrastructure or shared services. The scenario I often see is that rather than defining who/what/where the market is and providing a solution that addresses that need, the approach is more often "How can I convince those people to use what I have or want to build".

To me the difference is the lack of a product development team (in the marketing sense). Once that is in place then they must work in conjunction with engineering.

It seems that even Microsoft sees the need to improve this model within their own organization. In the course of developing the next release of Exchange, the two departments wrote a book together. Bink.nu: "It encompasses everything from the market outlook to the perceived value of possible features to potential customers. At its essence, the book serves as a contract between marketing and engineering, describing what goes in and what stays out of the software."

This seems like an approach that more should model.

Can Google help Firefox?

Steven J. Vaughan-Nichols in his artcile Firefox Is Heading Towards Trouble said, "If the Mozilla Foundation and Firefox friends like Google don't start spending money—right now—to hire more programmers, more project managers and more servers, it won't matter how many ads in the New York Times Firefox supporters take out, Firefox will have already reached its high tide of popularity and we can only wait for the ebb to begin." I think he has a good point. There has been much speculation about Google and the gBrowser. What there are up to is certainly not clear. But it seems a pretty good bet that they are doing something with the firefox code base. Although this may dilute the market share for pure firefox, I suspect it would help a lot with code quality, and may even server capacity.

A good overview about the benefits of workflow

I ran across this short note from George Parapadakis about workflow benefits. I thought it was a good summary. And You Know - Of Course What Workflow Is

Software Factories

I have been reading a bit about software factories, and also recently attended a presentation by Jack Greenfield (author of Software Factories: Assembling Applications with Patterns, Models, and Tools). Many interesting ideas.

The first thing to keep in mind, is that there is a war going on. The combatants are familiar: Microsoft and a few friends on one side, eveybody else on the other. As somebody once said, "the first casualty of war is truth".

One of the main principles of the software factory idea from Microsoft is that of "Domain Specific Languages". Fankly I am having trouble accepting the idea as useful for large scale developments in large development shops. I can see how it might work well with a team of A-Team programmers. But when dealing with 9-5 programmers - those with lives away from programming, I think having a dozen DSLs within the organization will be too much to handle. These DSLs will be on top of the general programming languages we already have. At least for the next 10 years. The average Joe doesn't want to learn that much.

On the other hand, I can see the counter argument. As Grady Booch points out, " In many cases, the semantics of the UML are pretty close to what you need" (December 3, 2004), UML is not exactly what you need - just close. There is often an impendance mismatch - which means you lose power, perhaps introducing distortion. The DSL approach says create exatcly what you need and use that - the mismatch is avoided.

The difficulty I have with that is that it may just be moving the problem. Now the programmer has to deal with a dozen different interfaces (DSLs). Is the programmer more likely to make a mistake in the coding? Would there be 'Whoops - that would have been the way to code it using DSL#34, but in DSL#93 it is done this way, becuase of what we learned when we used to use DSL#67. "

Grady also said, "we do disagree with Microsoft's rejection of the UML in favor of proprietary domain-specific languages". At first is wondered from where he pulled the word "proprietary". That word to me usually implies 'closed', 'owned', 'not generally available'. I thought these DSLs could be open, public, generally available. But then when I consider the number of them that could exist, I realized that proprietary is not so wrong.

Enough for now. This Software Factories versus MDD (or MDA) battle will be ongoing for years. I don't either will win, but both will declare victory.

Java over-engineered?

I don't know the name of the author of this post, but he (or she) has some interesting observations. About two-thirds of the way down the post he has a list of things that are bad about J2EE - in comparison to .NET.
What I think is most interesting is that I think he is comparing typical solutions built on those technologies rather than J2EE and .NET. Ya I know - semantics.
Anyhow, he goes on to blame this on "cross-platform compromises" and "textbook OOP".

Hmmm.

In the many commercial grade J2EE code bases that I am familiar with (high volume web banking and brokerage web sites), I can't say that we have made many cross-platform compromises - very few in fact. But when we do, we usually encapsulate the dependency, and put a factory in front of it. This add layers and makes the code more opaque. That is certainly true. But, as I said, we don't do that very often. Would stored procedures made the code simpler? I don't think that would be true either. In our case, it would add another layer!

What about textbook OOP? Is that a bad thing? Must be. I am wondering if what is observed is a tendency to make things overly abstract, overly decoupled - perhaps even overly reusable? The motivation is good - planning for change. It is often done with a view to save money in the future, without looking at the ROI. I think that too often we spend two dollars today to save a dollar next year.

.NET does not have any redeeming features that will save it from the same fate - (except maybe an army of VB6 programmers who haven't seen an OOP textbook!). Given time it will build it's own empires. Perhaps with the help of Microsoft! Chris Anderson and Miguel de Icaza exchanged some volleys about the complexity of Avalon see here and here.