Terry's Technology Topics

On UML as a Modeling Standard for Internal IT Purposes

2011-02-01T11:37:00.000-05:00

UML has evolved into a very large and fairly comprehensive Standard (http://www.uml.org/). The size and complexity allows it to be applied to most IT problems and also can create problems of it's own.

I do believe that it could be used to solve some of the problems I see in my day-to-day job and, with proper application, solve some of the problems I face.

'What we've got here is (a) failure to communicate'

- from 'Cool Hand Luke'.

Communication problems are the crux of the matter as I see it and come in two basic forms; challenges between two or more people trying to communicate an idea or situation at a point in time and also over time. In the first case there is an opportunity for the two parties to interact and discuss. In the second case that is not typically the case; a document is created and then read months or years later. The original author may no longer be with the company.

Aspects of the problem I see are:

Integrity (correctness and completeness),
Comprehensibility (succinctness and semantics),
Usability (navigation and accessibility)

Before I go into these in more detail and describe how UML may fit into the solution, I need to define a scope of applicability - the context within which I am think about this. In my current job I deal mostly with IT investments, processes, business systems (solutions), and requirements. I don't often deal with software architectures (the deep structures of software inside those business solutions), or the deep details of deployment. I also usually only deal with the results of the business requirements process and not the detailed creation of them.

Very often the results of my work (or of a team of people with whom I work) is depicted in the form of diagrams within a PowerPoint. Although the diagrams titles are used over and over again (business context, system context, and anything followed by the word architecture), there are no common rules about what goes into them, no common definitions for what lines and boxes mean, and very little consistency across teams, or time. My common tools for creating and managing this information are: Visio, Excel, Word, PowerPoint, and Sharepoint. All though we do get a lot done with these tools, there are challenges with the integrity, comprehensibility and usability of the resulting documents. Let me illustrate with a Visio example.

A box likely means something exists (physically or conceptually) and a line means that two things are somehow related. The precise meaning of the diagrams parts is defined by the author and may not be included in the diagram itself. This description might sound like total chaos, and that would be misleading. People within my organization have learned what those things usually mean within the context they are presented and can deal with the ambiguities and information gaps that exist. In most cases, the fact that a diagram abstracts a complicated physically deployment and represents it as a simple line is useful.

For example, I might depict the communication pattern between two applications as a simple line and maybe I put the label MQ on it. That tells many of my peers quite a bit. If they also know that those two systems are deployed within our own datacenter and a vendor datacenter then they can likely guess that there are likely two queue managers involved, a few MQ channels, several servers, likely at least two firewalls and a likely more IP switches than we would care to document. However, if I were the network engineer, I likely care about each and every one of those switches and circuits and not much at all about the communication activity between the two applications. Until there is a problem, or we need to make a major change and at that point all parties are interested in ensuring all levels are understood.

So we need to be able to communicate the idea that systemA talks to systemB and we want to show that as a simple line. We would like to be able to look inside that line to see the MQ specific characteristics and topology. Drilling further down we would see IP socket level details, IP addresses and ports - useful to network engineers and people writing firewall rules. Below that there is even more detail, circuits etc. Of course these layers of abstraction have been documented in another standard reference - the OSI 7-layer model (http://en.wikipedia.org/wiki/OSI_model). Should are architectural diagrams follow the same layers? Perhaps. Perhaps we should be producing artifacts that align with TOGAF 9 (http://www.togaf.com/)? Or Zachman (http://en.wikipedia.org/wiki/Zachman_Framework)? This is a major outstanding question for me. Maybe there are other possibilities.

Some of these frameworks provide better guidance at that than others. Zachman, for instance, provides guidance on many ways we should partition out models. TOGAF takes a different approach and is a little more concrete in some areas, but is generally less specific about deliverables. Which ever framework is chosen, I am thinking the most important part of the application of UML to the problem is getting guidance that is specific enough that two people would produce a models that are similar at the semantic level. What do I mean? If I were to point to people at a running system including all the code, deployment descriptions ands operational documents, and told them to draw the UML diagrams that describe the system at OSI layers 4 through 7, would they produce diagrams that use the same symbols in the same way? Would they define the same stereotypes and make the same profile extensions? I don't think so. There does not seem to be a standard methodology which can be applied to UML that gives good guidance on this.

I recently was involved in the review of a large number of infrastructure diagrams that were produced by a team. It wad quite clear that the individual artists had different ideas and rules in their heads. One person would focus on the network topology, another on the data flow (process oriented), and yet another on the IP level session. Production and disaster recovery (DR) path were always on the same diagram, but sometimes the DR paths were dotted or pink, sometimes not. Sometimes you could identify active-active clusters, sometimes not. If we were to re-execute the same task using UML and a common UML tool, would it be any better? I don't see that it would be. The symbols would be more consistent, but beyond that I don't expect much improvement.

It may actually be worse. When you have a free-form Visio diagram you know that there is no defined semantics for the graphical objects. But once you put them into UML and then into a repository you might think you know things we more confidence than you actually do. Garbage in, Garbage out. I have seen the progression from visio diagram to excel spreadsheet and then aggregate all the spreadsheets into a database and run queries. Once we have gotten to that point we are in danger of drawing conclusions from data that was shaky to start with. When it was a visio it was unstructured and it was apparent that only so much could be done with it. Once it becomes more structured (without adding knowledge in the process) one might lose sight of the inherent weaknesses in the source.

So back to my MQ example. In its simplest form this is two boxes and one line. Should the two applications be 'components' in a UML model. Should the MQ connection be a 'Usage', a 'Component Realization', an 'Interface Realization', or an 'Association'? Or something else? As I have poked around I do find answers, but I am left to think that somebody else doing the same job might get different guidance. When we try to aggregate our work together at some future time we might discover a large amount of rework ahead of us.

Is there an undiscovered part of internet that holds the answer to my quest? Leave a comment and let me know what you think on the topic.

Goliath and IT Security

2010-12-04T10:08:00.000-05:00

Regardless of whether you hold any special significance to the Judeo-Christian scriptures or not, I suggest that there is wisdom to be gained from their study. A large number of topics are covered in these ancient texts, including IT Security.

Yes, IT Security. The story is told about a long running battle between the Philistines and the Israelites (1 Samuel 17 1-53). Everyday the Philistines would send out their champion, Goliath, and challenge the Israelites to a one-on-one battle. Winner takes all. The big problem was, and I do mean BIG, is that Goliath was a giant. One big dude. No Israelite solider knew how to fight such an enemy. So they cowered. The philistines laughed and were comfortable.

One day this kid comes along bringing lunch for his soldier brothers. His name was David. He is witness to Goliath's daily challenge and like most young idealistic people, who can solve all the world's problems, he says, 'yo bros, why aren't we doing something'! They laugh him off. But David persists and soon enough finds himself facing the giant; Armed with the tools he knows best. Goliath laughs and mocks him. David splits his skull with a rock from his slingshot. Fair trade. Goliath dead, the Israelites win and the Philistine army scurries away.

So how do I get from an ancient military battle to IT security? Goliath was proud, he thought he was invincible and so did the rest of his army. He likely was invincible against all attackers who engaged him in the defined model of combat: sword, shield, spear. David didn't play by the rules, he even tried on standard armour but quickly realized it was not going to work for him. Instead he thought outside the box and made use of tools that were not expected on the battle field and won.

All too often in the IT context I hear people say things like: 'our software is perfectly secure', 'It cannot be hacked', 'I signed to SOX attestation so I know that security cannot be circumvented'. Ya - and David can't kill Goliath. We all too often fail to recognize the difference between the truth regarding what is possible and our ability to conceive of what is possible. We only truly know the bounds of the possible once we have exceed them.

If you are convinced that something cannot be hacked, broken, violated, or circumvented then what you are truly admitting to is the limit of your imagination. And never forget about the possibility of the $5 wrench.

Note taking applications.

2010-11-27T19:06:00.000-05:00

I have had my iPad for about 6 months now, and it has certainly changed the way I use computers. It has become the one device that I use the most. Even more than my work laptop - but that has a lot to do with the amount of time I am spending in meetings these days.

In meetings I do use my iPad for note taking. The most common application I use for that is Evernote. I like that it automatically synchs my notes back to my computer. I have been taking monthly subscriptions to the premium service so I can have offline notebooks as well. The one short coming I find is that I cannot draw with Evernote. So I have started looking for another program to fill the gap.

I have been using Adobe Ideas for a bit as well. It is good. I like the zoom in and out capability. It also smooths out jittery lines nicely. But it does not work so well for written notes. The new version allows for emailing of completed drawings. I would like some better interop with Evernote.

With Evernote's Trunk feature there was an app called Inkest. I gave it a try. It seems to be geared towards artist who would like to sketch. I didn't find that it worked well for written notes. It does allow for drawings to be saved to Evernote's web site (not the local app). If you are connected that doesn't page much difference, but if you are not connected you will have to remember to Upload your notes latter.

On to my third try. InkShelf. It has a nice notebook interface similar to iBooks. Some templates that help get things organized. It does have the nicest writing experience ... The ink flows really nice. The multipage paradigm within a notebook is nice. It does allow you to pick different papers for your notebooks, which is nice, but it is a choice for the entire notebook, so if you want lined paper on one page and grid paper on the next, tough. It does allow you to save your notebooks to both Dropbox and Evernote.

There are two features from all of these that I am still looking for. The ability for the app to transcribe cursive writing into typed notes and the ability to edit the notes once they have been saved to a cloud mechanism.

I have used penultimate on other peoples iPads. It seems nice, but I think I prefer InkShelf.

The next thing I need to get is a stylus.

Free Music at Walmart

2010-08-23T19:02:00.000-04:00

I just downloaded six free songs at walmart. Check it out.You'll need a walmart account.

Camera Connection Kit.

2010-08-02T15:15:00.000-04:00

I have been giving the Camera Connection Kir for IPad a pretty food work while on the road.

So far I like the capabilities. But I do have some gripes. I am hard to please.

First, it works as expected. I am working with CF cards from my Canon. All shots are RAW. It can see the RAW images and import them, but I think it is just grabbing the embedded jpeg. I don't know for sure.

Things I would like to see improved. I'd like to be able to:
- know how much space I am using. There is an app for that but it should be native.

- create albums and place pictures in them.

- edit places.

- crop, straighten and adjust.

- import videos (avchd) - may support others, but not my cameras format.

Overall I'd like "Lightroom for iPad". That does all of this. It should work in concert with the regular Lightroom as well. For example, import embedded jpegs (or better) into iPad, classify, assign metadata, titles, create collections, etc. Then later import same images as raw from card into lightroom for the desktop and have lightroom for iPad provide an update feed.

There are apps for many of those features, but it would be nice to have it all in one app. "Lightroom of iPad"?

I also did some two image HDR processing on the ipad using TrueHDR. Very easy and good results. See here.

TrueHDR sample

Happy Canada Day ... The Canadian Bookstore Arrives

2010-07-01T11:22:00.000-04:00

Previously I lamented about the lack of selection in the iBook store. It was just the free titles at that time. Today the Canadian content came online and there is a lot more to choose from. I still find the selection to be thin.

One thing I didn't notice before was alerts. You can set an alert on an author so you notified when a new book by that author becomes available. It would be nice if you could set an alert on a search phrase.

A feature I like about the amazon store is the wish list. Sometimes I am interested in something but don't want to buy it quite yet. Amazon let's me make a note about that by adding it to the wish list.

Zinio - I like it.

2010-06-26T17:18:00.000-04:00

One of the apps I decided to try out based on somebody else's recommendation was Zinio. It is a magazine reader much like iBooks is book reader. There is a fairly wide variety of magazines available, but not as complete as I would like. They do offer a few complimentary issues which allows you to check out the experience without having to subscribe to anything. The app itself is free.

I browsed through the compliementary issues and was impressed enough that I have subscribed to a two. The text clarity is excellent, images are good as well. These magazines are more than just the print version render in electronic form. The ones I have explored have embedded videos, and slide shows. It is generally an engaging reading experience.

One quibble have will point out is the app is not the most stable. I have found it to crash about every 10 to 15 minutes. Long enough between crashes to be tolerated but definitely annoying. Fortunately it does a pretty good job of saving where you left off when you restart the application.

I have not done much to explore oth magazine readers. Have you? If you have found a good one leave a comment to let us know.

Thoughts on iBooks

2010-06-24T12:49:00.000-04:00

Perhaps it is because my iTunes account is Canadian, but I have found the content a available on the iBooks store to be very disappointing. Close to useless for anything except old classics. I need to read some of them anyhow. Hands down the selection on Amazon is way better. However, I do think that iBooks is a better book reader than kindle. Oh well. The kindle read works quiet well, but iBooks just has a little extra polish.

What kind of polish? The bookmarks are nice as is the position within chapter and book. It seems less abstract than kindle. The page turn animation gets a lot of oohs and aaahs, but I am not so sure how much better it really makes the reading experience.

The one thing I really like is iBooks PDF feature. I have tons of PDFs; manuals, books, articles that I either want to read, or would like to have handy. A great example of a manual I have placed in ibooks is the manual for my camera.

Do you have a dropbox account? If you don't consider getting one. The drop box app makes it really easy to grab a PDF from drop box and then add it to iBooks. Slick. Get your own account here.[full disclosure - if you use that link I will benefit by getting more space. So please use it ;-)]

One big question remains - i mentioned it awhile ago when I wrote about the kobo. Just how many electronic bookshelves do you want to manage. I think I would prefer one shelf, connected to the amazon store which I could read with iBooks. Unlikely in the short term I think.

One improvement I would like to see to iBooks itself is a better way to manage a large collection of books.

Flash - my verdict

2010-06-23T07:46:00.000-04:00

Much has been written about this by others so I'll just recap my experience. Are there websites I visit that use flash as an essential part of the site's purpose? Yes. Would I like to visit them on my iPad? Yes. So, if they want me to visit, they'd better change. I think that goes for my 20-something children as well, who want to borrow my iPad rather than hauling out their laptop.

Ironically of the the sites that I really wanting to use, bad enough to go to my windows machine is the Adobe store. It is all Flash based. (and not well designed either, but that is another story). I have been unable to complete my order there, I may buckle and actually call customer service, but only because I want the product sooner rather than later. If it had been available on amazon for immediate ship, it would have been bought already, and done from my iPad.

New family member - welcome iPad

2010-06-22T15:12:00.000-04:00

I finally broke and bought an iPad. My rationale is that it does appear to be defining a new class of device and even though the next version will be much improved, I needed to understand the device now. For me that means using one as much as I can. Those who know me know that I will use it heavily for as much as I can.

I have had it now for a week; 32GB wifi. My rationale for the size was that it was bigger than my 16GB touch, and I could save a few dollars over the 64 model. Wifi only as I anticipated that if I really need mobile data access I would get a MiFi. I was also hoping that I would be able to pickup the hydro-one wifi signal from my desk. They offer an iPad data service for $5 per month. Unfortunately, I have not been able to make that work. Free open access does exist in many places downtown so generally am not too bad off. I think I will still miss the gps function.

I also purchased the dock, the VGA cable and the camera connection kit. The later because I hope to experiment with it as a tool for photographers as well.

So far the experience has been good. There was a good chance of it's new life with me when it was not available. Maybe I should have ordered two - or three. It has been very popular with the family.

That should be a good start. Of course this is all written on the iPad itself using the blogger web application. Works well.

The iPad and TV?

2010-04-11T16:51:00.000-04:00

In this post, Dave wonders if the iPad will kill cable TV news. In my opinion they are at severe risk if they don't respond in some way.

I rarely watch TV to begin with, but when I do, I take my iPod Touch. It won't be too long before that becomes my iPad - unless my wife steals it.

My kids are more likely to watch tv on their laptops than on our tv.

My conclusion? Cable news out, fiber to the home in.

International Kindle not shipping until Oct 19th

2009-10-07T09:31:00.001-04:00

See the Bloomberg article. http://www.bloomberg.com/apps/news?pid=20601087&sid=aiywhz5s9L7g
Given that info it is not surprising that Canadian orders are declined. Although Amazon should have thought about accepting pre-orders.

in reference to: Canada snubbed as Kindle goes global - The Globe and Mail (view on Google Sidewiki)

Sidewiki - kind of cool

2009-09-25T20:59:00.001-04:00

Since I like recursion, I'll make a sidewiki entry about my first sidewiki usage. Of course I'll post that to my blog as well.

I wonder if they will tie this into Google Wave?

in reference to: Terry's Technology Topics: My first test of sidewiki (view on Google Sidewiki)

My first test of sidewiki

2009-09-25T20:55:00.001-04:00

Not that this is a race, but I did need to beat Tom to the punch on sidewiki. Just because. It may be an interesting and useful tool. Lets see how it goes.

For this first test, I'll share the entry to my blog as well as Twitter.

in reference to: Google Sidewiki (view on Google Sidewiki)

Brilliant Attacks

2009-07-10T09:08:00.002-04:00

Brilliant attacks. (especially the laser one !)
- Monitor the powerline and detect and decode keystrokes from 15m away
- Monitor vibrations on desktop object via laser reflections and decode keystrokes.

The more secure we think our systems are, the more we must remind ourselves that there are styles of attacks that we cannot conceive.

If anybody tells me ‘this is complete secure, it is unhackable’, then this I know for sure about the speaker is that they are NOT very imaginative. They might not be thinking very hard about possible attacks. To me this makes products from vendors who make such claims less secure, they will be blind-sided by some hacker with more imagination.

What is Scalability?

2009-07-07T09:15:00.000-04:00

In computer systems we usually mean "linear scalability" and describes a relation between two measures. It is typically a reference to capacity. Two common measures are CPU capacity and number of users. If an application supports 10 users with 1 CPU and 20 users with 2 CPUs and so on until the numbers get quite large, we would say that it is scalable. because "number of CPUs" = 0.1 * "number of users".

It is a bit more complicated than that because we often have other constraints such as "user response time remains constant" and memory use scales as well. My first example also ignored the possibility of a constant offset, which would represent some fixed overhead. perhaps one-half of a CPU is required even if there are no users. For those who remember their math, y=mx+b is the equation which describes a line.

In my first paragraph i mentioned a possible exception "until the numbers get quite large". How large is large? That depends on the situation the solution is in. We typically would consider sizes that are significantly larger than what we expect, but still within limits. These limits can vary by the nature of the solution. If a workload is driven by internet stock trades, our degree of growth we could expect is much more volatile than if it is the number of Canadian branch locations for a bank (which is pretty much already saturated).

It would be false to assume that the linear scale continues without bound. Linear scale without bound, would be a truly rare situation. At various points as workload grows you will run into "walls". They are called that because when you look a the graph of this situation your resource usage grows much faster than your workload - as if it hits a wall.

These wall situations usually occur because some other resource becomes saturated. For example, perhaps your database server "maxes" out. In that case adding more CPUs to your application server won't help. But, perhaps re-engineering the database server will remove that constraint and allow for further growth. Sometimes these walls are "hard"; that is re-engineering won't alleviate the constraint.

There are two sources of these hard walls: coding and architecture. Some may argue that "coding" is just a different type of engineering constraint. I won't argue that, but in my company we have an engineering department that specializes in server sizing and configuration, and development departments that do the coding. so we classify them as two distinct problem types. I have another reason for that differentiation as well. Engineering constraints can usually be quickly fixed with the addition of more resources (server, CPU, memory, etc) or reconfiguration/reallocation of existing resources (add more threads, connections, heap). Ccoding problems on the other hand usually take much longer to diagnose, recode the problem area, retest and redeploy. A trivial example of this would be the replacement of a linear search with a hash table lookup.

Architectural constraints are more fundamental design decisions which can not easily be altered. For example a design decision that requires an application to execute completely within a single server. This might be a simple design that performs well - as long as you can buy a larger server. whether this is a good decision or not depends greatly on how reasonable your assumptions about the potential for growth may be.

Is scalability always a good thing? Perhaps not. it depends on what you are measuring. I recently read a product evaluation that said (incorrectly) that the product's license model was not scalable. The truth is that it is high scalable. The more of the product we used, the more we paid. It wasn't linear though, because volume discounts meant that unit costs dropped as volume increased (and that is a good thing!). What the author really meant was that they wanted NON-scalable pricing; they wanted a price ceiling (somewhat like a wall except on the other dimension). At a certain point of volume growth, they didn't want to pay anymore. a desirable feature for the buyer, bit maybe not the seller.

There is much more that could be written, horizontal versus vertical scaling. 'knees in the curve', etc. But until then you might like to read the wikipedia article.

8,000 US Banks

2009-04-20T10:32:00.000-04:00

"the U.S. still has over 8,000 banking companies": Anybody who wants to understand the US banking industry need to understand this point. The referenced article gives just one viewpoint on the degree of diversity that is 'The US Banking Market'.

Wacky pricing... paper, ebook, and audiobook

2009-02-13T09:40:00.002-05:00

Something is just plain wrong here. I have been listening to Taleb's "The Black Swan: The Impact of the Highly Improbable". A very interesting book, by the way. Recommended.

Anyhow, this is about price. I bought the audiobook from Audible.com so I could listen to it during transit times, etc. I paid C$15 for it. Expensive for a download I thought, but Amazon wanted US$21 for their audio download. The audio CD? US$26 !!

I decided to check out what a paper copy of the book would cost. I wasn't surprised to find it on amazon.ca for more money, but not too bad at $20 for the paperback. But I don't have any more space on my bookshelf.

So I looked for an ebook. I was shocked to find the price to be US$27 at several sites. That is insane. Finally, what about amazon/kindle? US$12. Much more reasonable, but you have to buy a Kindle. Sony's price is the same.

I think there is a bit of room here for price competition. I can't see any reason why the electronic download should not be universally half-price compared to the original media. This is somewhat the way it is for music. Perhaps libraries provide price competition on the paper?

One other thing I noticed. It doesn't seem to be general practice for a retailer to offer multiple media formats for an item - the one exception being the Kindle view on amazaon. Surprisingly the reverse was not the case. The regular amazon entry for the hardcover book did not reveal the other options available. A missed opportunity.

Prototype, Proof-of-Concept, and Pilot. Oh my!

2009-01-09T09:16:00.001-05:00

These terms are often used in IT contexts often without much consideration to nuances. Whatever you call it, the first important thing is that you understand and state your objective. The second is that you meet it.

My suggested definitions.

Pilot: This is an implementation of a system that is often functional complete. It is typically deployed in production, but usually constrained to a small number of users. Although we hope everything is perfect, there is an expectation that there will be faults that require rework - otherwise we could have just gone full production. The fault maybe in deployment, code, design, or in user experience. A pilot is typically time-boxed. A pilot is usually fully productionalized from an operational perspective.

Beta: Similar in many aspects to Pilot. There is a lesser expectation that it is functionally complete, but it typically is. With a Beta, there is a much more explicit understanding that it is not final. It will change in the final release. A Beta is often supported, if at all, by a different organization than a production instance, typically developers. Traditionally a beta was not to be used for production, however some companies are making it part of their normal process - the never ending beta. Something that Google has done many times.

Release Candidate: A software build and might be view as living in the middle between Beta and Pilot. A release candidate may be promoted to pilot or production.

Proof of Concept: I view this as a very narrow and well defined activity. There is a well defined concept. The objective of the activity is to prove that the concept is viable in some aspect. Functionally it is only complete enough to meet the objective. The resulting code is not intended to be used for anything else - although most programmers will harvest some aspects for other things. Agile methods talk of 'early pain'. Significant projects has technology aspects that well be challenging and be a source of risk. It is desirable to execute on those aspects first, if there is a problem you want to know about it early so that you can change your plans or maybe cut your losses. This is what a proof of concept is about; if you are going to fail, fail early. Many times I have seen people propose a 'proof of concept', with no idea what concept they wish to prove; often what they mean is that they want to start coding. PoCs are not implemented in production.

Prototype: Perhaps this one has the most varied definitions: Experimental Prototype, Engineering Prototype, etc. In software development, a prototype is a rudimentary working model of a product or information system, usually built for demonstration purposes or as part of the development process. As part of an SDLC approach, a simple version of the system is built, tested, and then reworked iteratively until ready for use. Prototypes are not usually implemented in production. Go read the wikipedia article http://en.wikipedia.org/wiki/Prototype

Do you agree? Are there any characteristics of any of these terms that you think would help define them.

Unstructured, Semi-Structured, and Structured Data

2009-01-05T13:27:00.002-05:00

I originally wrote this over two years ago and have intended to post it ever since. Not too late.

Often in the IT world we hear or even use these terms. But what do they really mean? Here is my view.

All bits and bytes that we deal with in the IT world we considered to be data (at least). It all has some form of syntax and structure, so what do we really mean, and why is it useful to distinguish between them?

These three classifications represent a continuum which spans from unstructured to structured data that represents the degree to which the data's semantic model (meaning) matches our processing requirements. In general what we are trying to describe is the readiness of the data to be processed in a particular business context .

For example, if the data in question is the raw audio recordings from the call centre, and the business context is we need to review all verbal instructions spoken by customer "Joe Smith" last year over the phone, we may consider that recordings to be unstructured. We have no easy way to process the request.

If we have augmented those recordings with additional data from other systems and have added customer number and call timestamp to the recordings (or an index) then we would consider that to be semi-structured data. Although we could quickly sift through the millions of minutes of recordings to get Joe's subset, somebody would still have to listen to the recordings to find the things that Joe said.

The structured data, in this case, could be represented by the actual transaction records that the call centre agent created in response to Joe's instructions.

Likewise, A TIFF image might be considered structured data within the Context of a GIS application (geographic information system), but might be considered unstructured within a mortgage appraisal application. (Perhaps even GIS would consider it unstructured since they might ideally wish to run queries over an image set to find all lakes larger than a certain size. That would be hard on untagged TIFF images.)

All the data we typically deal with has a known syntax, even if that syntax is only really understood by MS Word. And although a Word document may have semantic meaning to a human, that semantic meaning is not easily extract by computer. We consider a Word document to be unstructured (in most cases).

An Excel spreadsheet may have a well defined layout of rows and columns. Although Excel may find it easy to 'understand' its content, other programs may or may not. If they layout is regular and complete, programs other than Excel maybe able to extract that data from the spreadsheet and do useful things with it. We would consider that to be semi-structured. I suggest that the 'semi' aspect of the term introduces the concept of a degree of uncertainty. Perhaps this is because it source is not well controlled and the form (layout) may change and it suddenly becomes unstructured in our context.

Structured data has an aspect of surety about it . We know that there are 'fields', we know where they are, we know what values to expect. We know how to understand it. We expect there to be some kind of formal model which defines this structure, and we expect that there will be controls in place that enforce our expectations. We may often visualise such data as being a relational form stored in a RDBMS. But that is not a requirement.

All that said, here are my definitions:

Unstructured Data: Data which does not have the appropriate semantic structure which allows for computer processing within a particular business context .

Semi-structured Data: Data which has some form of semantic structure which would allow for a degree of computer process within a particular business context, but may need some human assistance. It may apply some heuristics but the process may fail due to volatility of the structure or incorrect assumptions about the structure.

Structured Data: Data which is well positioned to be reliably processed by computer within a particular business context. It has a well-defined and rigorously controlled syntactic and semantic structure. The elements of the data have a well defined datatype and rules about valid values and ranges. The meaning of these data elements is well understood in isolation as well and their relationships to other elements. Elements are also traceable to their originating sources and that path is verifiable.

Synovus to merge Americus and Albany banks - Atlanta Business Chronicle:

2008-12-22T11:19:00.000-05:00

Synovus to merge Americus and Albany banks - Atlanta Business Chronicle:: Something doesn't seem right here. Consider these two quotes.

"Once completed, the combined banks will have $649 million in total assets in seven locations, Columbus-based Synovus said."

"On Dec. 19, Synovus received $967.8 million in funding from the federal government’s Treasury Troubled Asset Relief Program (TARP)."

They have received more in TARP funding than they have in assets!

Quantum cryptography unbreakable?

2008-10-08T13:11:00.000-04:00

Laser cracks 'unbreakable' quantum communications

I have maintained for years that there is no security scheme that is uncrackable it is just a matter of are you smart enough and perhaps have enough money.

If a vendor tries to pitch a product to me and claims it is completely secure indirectly also tells me they don't really understand security and that their people are not smart enough to even consider how to attack it and succeed.

Quantum crypto was hailed as being uncrackable. I didn't believe it even though I knew relatively little about quantum anything.

Finally remember the Maginot Line.

An opportunity to pick up either space or customers, or both!

2008-10-08T08:11:00.000-04:00

Bank Mergers Spell Change for Branches in Manhattan

Nervous customers may wish to jump to a competitor - perhaps seeking to diversify their deposits to fit within the FDIC coverage, and empty brnach locations provide plenty of opportunities - if we can talk ourselves into the spaces.

'Discomgoogolation' - this is why I have a blackberry!

2008-10-06T08:06:00.000-04:00

Feel stressed if you can't get online? You could have 'discomgoogolation'

I do hate to admit it, but while in Alaska and Yukon in July there were days on end when I had no internet connection. It was somewhat stressful. I guess I should do that more often.

Thinking into the future

2008-03-23T12:29:00.000-04:00

Be Like the Internet - 8 steps to success in a post 2.0 world

From: Thor, 10 months ago

| View | Upload your own

This is v1.0 of a presentation that Lane Becker and Thor Muller are workshopping. It was delivered for the first time at WebVisions in Portland, ... less Oregon.

SlideShare Link

Software Architect Guy ...

2008-03-01T07:31:00.002-05:00

A whiteboard in the shower. Brilliant.

Video: ODC2008 Architect Guy

If you like that, you may also enjoy "Greg the Architect".

I now have a New Jersey Phone Number

2008-02-26T20:24:00.002-05:00

Maybe not a big deal, but i find it interesting. I signed up for Grand Central - a google provided free phone service. I could tell you what the phone number is, but I don't want to publish it here, if you want to give me a call, you still can. click the button.

You will be asked for a number to call and my phone will ring.

I am neither architect nor engineer but an ecologist and gardener

2007-07-06T08:13:00.000-04:00

This discussion has been going on elsewhere, and it resonates well with me. It is useful to remember that all four terms are analogies. The thought space about my job as being a software architect or engineer are quite old, the ecologist and gardener ideas are refreshing.

Applicability of "Google’s three rules"

2007-06-25T10:07:00.000-04:00

Robin Harris wrote about "Google's three rules" for the data centre, in short they are:

Be cheap
Embrace failure
Architect for scale.

Great rules and they appear to be working well for Google. Does that mean that your organization should adopt the same rules? Not necessarily. As you read through Harris' discussion a couple of key success factors jump out at me that I suggest are pre-requisites of the Google approach.

"Free or home-made software": If you are paying license or support fees for the software you are running that has a huge impact on how you approach provisioning. For many Enterprise Data Centres (EDCs) I would expect that they have already discovered that the cost of the software stack can easily exceed the cost of cheap servers several times. That is very much approaching Gates' "hardware will be free" prediction. Schwarz said a similar thing as well. What remains that drives cost is the software stack, and if you want to put large multipliers in front of that cost, then it has better be a small number.
"Google hired some of the best minds in the business": rather than investing in software licenses or expensive hardware, they are investing in brain power. It seems to be a reasonable approach to me? Have they figured out how to scale brain power? Certainly there are lots of untapped talent around the world that should satisfy demand for a while - how long depends on how many companies take the same approach. The more significant limiter is not supply but rather their ability to manage the bureaucracy that will evolve. But this is not about Google, but your organization. Can you acquire brain power and scale it?

I am not saying the Google's approach is wrong - but if you don't satisfy those pre-reqs and forge forward with their approach, you will have problems. Lots little and yet expensive problems called servers.

Event Processing

2007-05-08T09:11:00.000-04:00

I've come upon lots of good reading lately on complex event processing. Interesting stuff indeed. Although there are some really cool things that can be done with this kind of technology, I think operational costsare a significant stumbling block for organizations like mine.

The article "Event Processing in 2007 and beyond" provides a good overview and some examples.

This site is dedicated to the topic and contains much good material.

Design patterns is a favourite topic of mine, and CEP Design Patterns provides a design pattern spin.

Multi-processor and Programming Languages

2007-05-02T06:36:00.000-04:00

"now that multi-core is in rapid adoption, the issue of multi-processor programming is beginning to loom large." I wasn't the first to say it, but it is reasuring to see others say it to.

With respect to StreamIt, it is a completely different programming model than the previously reported Fortress Language.

A legitimate alternative to passwords?

2007-04-19T09:17:00.000-04:00

| Tech Sanity Check: has a commemtary about yet another Multi-factor Authentication scheme.

vidoop has this interesting scheme for multi-factor. The interesting twist in this case is that the scheme has the potential to be based more on how you think than a known fact. You might select a sequence of pictures containing boats, airplanes, and cars. The theme could be transportation, the colour blue or aluminium.

They aren't there yet - it seems their current scheme relies on you thinking they way they do. But it does strike me as an improvement over static images.

Shared Items

2007-04-08T15:24:00.001-04:00

If this works out right, you should now see a shared items feed near the top of this page. What are shared items? Well many sites produce a summary of their content in a form that can easily be syndicated - that is consumed by other sites, or applications. Often referred to as a 'feed' or an 'RSS feed'. RSS is a particular protocol used to implement this feature. ATOM is another such protocol, although they often get lumped together and called RSS.

So what can you do with it? There are many ways by which you can consume these feeds. Internet Explorer 7 has such a feature built in. So does my.yahoo. My favourite mechanism is Google Reader.

If you know somebody who reads a lot of internet content, you may wish you could 'read over their shoulder' every time they say 'Hmm. That's interesting!' Well that is exactly what Google reader shared items let us do. If I find something if interest, I can mark it 'shared', and it will then show up on my shared items feed. If you'd rather just look at a web page of the same items, you can do that instead.

There is one thing I would like to be able to do, that 'shared items' doesn't allow. And that is to make a comment on an item I share. Well maybe we don't need that. That's what this blog is for!

Have great day.

Service Architecture - SOA: Five Nines in a Service Oriented World - Part 1 the problem

2007-03-22T13:13:00.000-04:00

Service Architecture - SOA: Five Nines in a Service Oriented World - Part 1 the problem: "Five Nines in a Service Oriented World"
A nice little discussion about the math behind availability.

Picking the Open Source Winners

2007-02-07T08:30:00.000-05:00

This is a big topic for me at the moment. I am very interested in the OpenBRR and Navica scoring methods and the work undertaked by ohloh and optaros.

Authentication, Authorization, and Context

2007-01-31T12:27:00.000-05:00

I was reading James McGovern's blog today and it reminded me of a conversation I had at work yesterday. James focus is on vendor product - something that is certainly of interest to us, but beyond that we also have to deal with our internal applications.

There are three issues that always pop-up when we try to integrate a new software product.
How do we authenticate?
How do we authorize?
What was the user try to doin the first place?

We are starting to get a good handle on #1 - but I would like to see us authenticate at fewer points and establish a trust network between applications. SPNEGO, SAML, WS-Federation, Liberty, and perhaps OpenID are all promising.

The James' blog speaks to the second part - authorization - The next big frontier. What roles can the principal hold (for this application)? The application part of that sentence is ways controversial. This is where XACML fits in.

The third piece - what were we doing is a tongue-in-check reminder that the user was actually trying to do a job before security "got in the way". The user likely had some kind of established context that should, ideally, be available to the next application. Although not always the case, it is a frequent requirement. For example, the user may have been working with a customer in the CRM application, and now needs to work on the customers Loan in the credit management application. This customer and loan context information needs to be carried forward. There are no good soutions for this that I know of. This is the undiscovered country.

Any thoughts out there?

Will we need a Fortress for our cores?

2007-01-19T08:29:00.000-05:00

Two news theme over the last several months strikes me as mutally relevant.

First there is the information coming from Intel about future core densities. The Gulftown processor will likely make its debut in 2010 and will contain 32 cores. Intel research is also working on an 80-core research prototype. As the InformationWeek article discusses, software will need to change to accomodate.

Speaking of software, Java is a significant development language for the business world. What will business applications do with an 80-core engine? Multi-threaded applications - wherein the threads are explicit to the programmer are not the way to go. To the typically programmer I say "You can't handle the threads".

So what is the second news theme? Fortress - a research effort coming out of Sun Research. It is targeted at High Performance Computing; A replacement for Fortran. Am I suggesting that we rewrite all of our applications in Fortress? No. But Sun is implementing Fortress on the Java Virtual Machine. So there are some good possibilities that future Java language features will be able to drive out the same multi-threaded behaviour for which Fortress is striving.

Bank loses data

2007-01-18T16:09:00.000-05:00

Bloomberg.com: Canada: "CIBC's Talvest Mutual Fund Loses Client Data Files"! This cannot happen too many more times before companies will be forced to put the technology in place to ensure that data that leaves their premises is encrypted. That won't be cheap, but it seems that the cost is inevitable.

Historical revisionism

2007-01-18T15:07:00.000-05:00

Historical revisionism - what does this have to do with technology. Well last month I was trying to figure out how to make Sametime display my picture. I found a very informative article about exactly that process. I thank the author for publishing the information.

However I was shocked by how difficult the process is. Perhaps suitable for the corporate world, but still awfully painful - especially when you compare that process to the drag-and-drop simplicity of MSN Messenger.

I made a civil comment on the topic - it seemed that the author and his readers may very well be infleuential in improving the process. I thought I was being helpful. I kept checking back to see if anybody responded to my observation. Sure enough. Today I discovered that my comment was deleted!

Now this is not a great travesty - certainly not in the same league as what the wikipedia artcile covers. And not the first time it has been reported in the online world. New York times artciles and Whitehouse pages have suffered the same fate. But it is the first time it has happenedto me.

But this is the internet, and this blog is the solution. Now you all know that changing your display picture in sametime is very difficult - and I won't delete your comment - unless it has something to do with Russian brides ;-)

The future of the ESB

2006-03-20T09:53:00.000-05:00

Loek Bakker's weblog: Understanding the future of the ESB: "It's so simple: ultimately, when all the parts have the ability to work together (i.e.: they all can communicate through messages, which will be the bus), the role of the ESB will be like the role of DNS for the Internet: addressing and routing. Nothing more, nothing less."

I suggest that he overstates the role of DNS. It doesn't really do routing. That is the job of routers and switches.

I generally agree that the capability we want for the end-points to do the processing, so that the actually communicaiton flow is point-to-point. But that point-to-point connectivity is not exposed to the application. I want to be able to modify application behaviours and policies as if it all went through a central hub, but don't want to actaully make a network hop and who-knows-how-many context switches in order to perform any routings and transformations.

Using DNS as an example, browsers look-up (resolve) domain names only occasionally, and then direct their traffic to the end-point directly. In some cases the end-point may do some virtualization and wrokload management at the far end, but all of that is of no consequence to the requestor. This is the way the applications should work with an ESB.

Tim Bray On PHP

2006-02-21T13:13:00.000-05:00

"Spaghetti SQL wrapped in spaghetti PHP wrapped in spaghetti HTML" Ouch. Tim certainly has an opinion here. I question where this has anything to do with PHP. One could certainly create the same kind of pasta using JSP and is the way many ASP sites look inside. I suggest the difference is not the language, but the focus of the developer.

Dan Bricklin's wikiCalc

2006-02-17T14:16:00.000-05:00

Dan was the original author of VisiCalc. He is at it again with wikicalc. An online spreadsheet. I like the idea, the alpha code is somewhat functional, but needs polish. Great work Dan.
Software Garden Products: wikiCalc Program: "The wikiCalc program is a web authoring tool for pages that include data that is more than just unformatted prose. It combines some of the ease of authoring and multi-person editing of a wiki with the familiar visual formatting and data organizing metaphor of a spreadsheet. "

Designing Software for Your Market

2005-06-02T09:36:00.000-04:00

One thing that bothers me is the way that many IT shops go about desginin infrastructure or shared services. The scenario I often see is that rather than defining who/what/where the market is and providing a solution that addresses that need, the approach is more often "How can I convince those people to use what I have or want to build".

To me the difference is the lack of a product development team (in the marketing sense). Once that is in place then they must work in conjunction with engineering.

It seems that even Microsoft sees the need to improve this model within their own organization. In the course of developing the next release of Exchange, the two departments wrote a book together. Bink.nu: "It encompasses everything from the market outlook to the perceived value of possible features to potential customers. At its essence, the book serves as a contract between marketing and engineering, describing what goes in and what stays out of the software."

This seems like an approach that more should model.

Can Google help Firefox?

2005-03-09T07:52:00.000-05:00

Steven J. Vaughan-Nichols in his artcile Firefox Is Heading Towards Trouble said, "If the Mozilla Foundation and Firefox friends like Google don't start spending money—right now—to hire more programmers, more project managers and more servers, it won't matter how many ads in the New York Times Firefox supporters take out, Firefox will have already reached its high tide of popularity and we can only wait for the ebb to begin." I think he has a good point. There has been much speculation about Google and the gBrowser. What there are up to is certainly not clear. But it seems a pretty good bet that they are doing something with the firefox code base. Although this may dilute the market share for pure firefox, I suspect it would help a lot with code quality, and may even server capacity.

A good overview about the benefits of workflow

2005-01-13T15:06:00.000-05:00

I ran across this short note from George Parapadakis about workflow benefits. I thought it was a good summary. And You Know - Of Course What Workflow Is

Software Factories

2004-12-06T08:03:00.000-05:00

I have been reading a bit about software factories, and also recently attended a presentation by Jack Greenfield (author of Software Factories: Assembling Applications with Patterns, Models, and Tools). Many interesting ideas.

The first thing to keep in mind, is that there is a war going on. The combatants are familiar: Microsoft and a few friends on one side, eveybody else on the other. As somebody once said, "the first casualty of war is truth".

One of the main principles of the software factory idea from Microsoft is that of "Domain Specific Languages". Fankly I am having trouble accepting the idea as useful for large scale developments in large development shops. I can see how it might work well with a team of A-Team programmers. But when dealing with 9-5 programmers - those with lives away from programming, I think having a dozen DSLs within the organization will be too much to handle. These DSLs will be on top of the general programming languages we already have. At least for the next 10 years. The average Joe doesn't want to learn that much.

On the other hand, I can see the counter argument. As Grady Booch points out, " In many cases, the semantics of the UML are pretty close to what you need" (December 3, 2004), UML is not exactly what you need - just close. There is often an impendance mismatch - which means you lose power, perhaps introducing distortion. The DSL approach says create exatcly what you need and use that - the mismatch is avoided.

The difficulty I have with that is that it may just be moving the problem. Now the programmer has to deal with a dozen different interfaces (DSLs). Is the programmer more likely to make a mistake in the coding? Would there be 'Whoops - that would have been the way to code it using DSL#34, but in DSL#93 it is done this way, becuase of what we learned when we used to use DSL#67. "

Grady also said, "we do disagree with Microsoft's rejection of the UML in favor of proprietary domain-specific languages". At first is wondered from where he pulled the word "proprietary". That word to me usually implies 'closed', 'owned', 'not generally available'. I thought these DSLs could be open, public, generally available. But then when I consider the number of them that could exist, I realized that proprietary is not so wrong.

Enough for now. This Software Factories versus MDD (or MDA) battle will be ongoing for years. I don't either will win, but both will declare victory.

Java over-engineered?

2004-09-22T07:27:00.000-04:00

I don't know the name of the author of this post, but he (or she) has some interesting observations. About two-thirds of the way down the post he has a list of things that are bad about J2EE - in comparison to .NET.
What I think is most interesting is that I think he is comparing typical solutions built on those technologies rather than J2EE and .NET. Ya I know - semantics.
Anyhow, he goes on to blame this on "cross-platform compromises" and "textbook OOP".

Hmmm.

In the many commercial grade J2EE code bases that I am familiar with (high volume web banking and brokerage web sites), I can't say that we have made many cross-platform compromises - very few in fact. But when we do, we usually encapsulate the dependency, and put a factory in front of it. This add layers and makes the code more opaque. That is certainly true. But, as I said, we don't do that very often. Would stored procedures made the code simpler? I don't think that would be true either. In our case, it would add another layer!

What about textbook OOP? Is that a bad thing? Must be. I am wondering if what is observed is a tendency to make things overly abstract, overly decoupled - perhaps even overly reusable? The motivation is good - planning for change. It is often done with a view to save money in the future, without looking at the ROI. I think that too often we spend two dollars today to save a dollar next year.

.NET does not have any redeeming features that will save it from the same fate - (except maybe an army of VB6 programmers who haven't seen an OOP textbook!). Given time it will build it's own empires. Perhaps with the help of Microsoft! Chris Anderson and Miguel de Icaza exchanged some volleys about the complexity of Avalon see here and here.

PHP versus Java

2004-09-01T13:55:00.000-04:00

Here is an intersting tidbit burried in a post about Friendster's switch from Java to PHP. As usual the debate tends to feed on how you build the application rather than the fundamentals of the underlying technologies.

Of particular interest was this quote. in the journal "ACM SIGMETRICS Performance Evaluation Review, Volume 31 Issue 3" there was an article called "A performance comparison of dynamic Web technologies" where Perl, Java server technolgy (also tomcat) and Perl was benchmarked in a labratory environment. It was concluded that Serverside Java outpreformed PHP and Perl by a factor 8. The abstract also states, In general, our results show that Java server technologies typically outperform both Perl and PHP for dynamic content generation, though performance under overload conditions can be erratic for some implementations.

Gartner says WSDL is a critical standard

2004-08-30T09:17:00.000-04:00

A recent Gartner Research Report, entitled 'Consider WSDL a Critical Standard'. Reinforced their opinion that WSDL and web services are a fundamental component to successful SOA. Their bottom line is use WSDL or find your self alone in an obscure corner of the industry.

More on Rich Clients...

2004-08-30T08:18:00.000-04:00

The IBM workplace technology seems to be gaining some traction. Stephen O'grady has written up an analysis of what some other vendors are proposing to do on top of it.

Actuate's plans are intersting for two reasons: 1)it is an open source business intelligence project, 2) it is built on Eclipse.

Rich Client Wars

2004-07-13T13:43:00.000-04:00

Much is being written today about the future of the Rich Client application versus the browser applicaiton. One side says you can't get a good user experience out of a browser application, and you can't get sub-second response time from it either.

There are those at the other extreme as well. The say there is nothing the browser can't do and notuing it shouldn't do.

I suggest that truth lies in the middle. I also think that the middle will shift over time - towards the browser side.

This posting at looselycoupled, by Phil Wainewright, talks about how the Yahoo purchase of Oddpost is an indicator of things to come - delivery of rich user interfaces within the browser. His main point is that Microsoft's Avalon is the wrong direction for the company to be taking.

He may be right. Time will tell. I think Microsoft has surely thought about that. I think they are hedging their bets - make the borwser irrelevant. Where you really wnat to work is in Microsoft Office (with a healthy serving of Sharepoint Portal Server and BizTalk server to help it out). Take a look at their Office Information Bridge. If Microsoft could get us to live in a purely MS-Office world, then the browser becomes irrelevant, as does Google, Yahoo, IBM, Sun, Java and Linux. Apple would still be around - after all it does run Office, and the Microsoft marketing department needs Macs to publish their material (look at the document properties for their PDFs - you'll often see something like "Mac OS X 10.3.4 Quartz PDFContext")!

Microsoft Project Competitor Released under Open Source License

2004-07-09T10:21:00.000-04:00

A company called Niku previously sold a product called workbench that was a serious competitor to Microsoft's Project. How serious? It sold for $10,000 a seat according to this InformationWeek report.

Despite the price tag, it was not a major revenue contributor, so Niku decided to release most of the code to the community under a Mozilla license. The product contains a few licensed components from others that cannot be released, as well, they will not release the source for their scheduling algorithm - which they may patent. They will distribute the binary for that piece. You can find out more at the web site.

Steve Ballmer commented in his corporate email that "Microsoft needs to do a better job of convincing customers that the latest versions of its products are worth having" (clipped from cnet). OpenWorkbench for free, or Microsoft Project for US$600 - Microsoft has its work cut out for it!

Interesting Items From Apple's World Wide Developer Conference

2004-06-30T17:40:00.000-04:00

I don't own a Mac. Somedays I wish I did. This week more than most. I would like to win the lottery too!

So what is so attractive this week. Well, first off there is the dual processor 2.5Ghz G5. It's not new. But it is what goes with it that is interesting. Like the 30 inch Cinema displays! Gorgeous. A pair of those would be ever so sweet.

What else? Spotlight. This is Apple's search technology that will be part of the Tiger release of OsX. Think of it as Google for your desktop. The demonstration at the conference was very impressive. Speed was excellent. But I think effectiveness it the more important aspect. With spotlight you can save a search query and have it re-executed any time. Because of its speed, one should regret opening such a folder. I know that when I do a search on my windows machine, I plan on it taking some time.

Finally there is Rendevous. This might come in handy within our branch environment for application deployments with minimal configuration.

These are each very cool. Good work Apple.

Loose Lips Sink Ships?

2004-06-17T08:35:00.000-04:00

Bruce Schneier recently wrote about the keeping of secrets in the world of international espionage. He discussed the American supposed cracking of Iranian encryption, and made the point that keeping secret that you know other people secrets can be very valuable.

During World War II, the Japanese made huge balloon bombs that they release into the air to be carried to North America. The intent was to start fires and kill people. It was of some success. However, 'In May 1947, the New York Times wrote "Japan was kept in the dark about the fate of the fantastic balloon bombs because North Americans proved during the war that they could keep their mouths shut. To their silence is credited the failure of the enemy's campaign."'

Somebody once said about music that the anybody can play the notes, but the rests make the music. Perhaps something similar can be said about security and silence.

Smart Dust, Even Smarter Vacuums?

2004-06-08T07:51:00.000-04:00

In case you haven't read about it, there is a technology called Smart Dust - glitter sized network computing devices that can do all sorts of cool and scary things.

So what happens after we produce billions of Smart Dust devices and spread them pervasively (invasively) throughout our world?

Perhaps this could spawn add-on features for our Robot vacuum cleaners?

Not only could your smart vacuum cleaner take care of regular dust, it could seek out smart dust as well? And then what?

Do we need virus scanners for our smart dust? Can we run Norton anti-virus on our Roombas in order to separate 'good' smart dust from 'bad'? Maybe we will need a Smart Dust recycling program?

A New BEA Emerges From The Beehive

2004-06-02T07:38:00.000-04:00

A few days ago i suggested that BeeHive was an important development to track. The META Group has published a report that is in agreement.

You need to be a customer to access the article. http://www.metagroup.com/us/displayArticle.do?oid=48577

California Passes Anti-Gmail Bill - Is this good

2004-06-01T08:07:00.000-04:00

I hate banner ads. Why? Mainly since they are such a waste of screen real estate. They are irrelevant. I click on a banner ad very rarely (yearly?), and when I do, it is usually from a web site meant for a targeted community, therefore the ads are most relevant.

I have been using GMail for a month now. The text ads that Google has been placing on the right hand side have been very relevant. I find myself actaully reading the ads, and clicking thru many times a week. Are they invading my privacy? I don't think so - since I agreed to it.

If there must be ads on the web sites I use, then I prefer it the GMail way.

BEA donates Beehive to Apache

2004-05-26T10:09:00.000-04:00

In October 2003, Crossvale (a research firm) published a report entitled “A Study in Enterprise Development Productivity”, subtitle “Comparing Implementation Approaches with J2EE API coding vs. Using a J2EE Development Framework”.

This report has been widely circulated by BEA as evidence that WebLogic is better than Websphere. However, in reviewing the report I found that they were really comparing the benefits of using advanced tooling to generate code rather than coding major portions by hand.

My conclusion was that BEA's WLW (WebLogic Workshop) may offer some improvements in our project delivery time, it also required a significant change in our infrastructure, and more importantly would require us to abandon our vendor neutral position.

On May 19 of this year, BEA announced that portions of WLW will be released to the open source community via Apache. The first code releases would be available later this year. It is a little early to gage the importance of this development. But given good market acceptance, this may be an importance addition to our tool set.

References
The Crossvale report
BEA Beehive Press Release
ZDNet News article

First Impressions of Gmail

2004-04-30T21:00:00.000-04:00

I have started using my new Google mail account. So far I like it. The interface is clean and simple. But the thing that impresses me the most is it's speed. It is very fast. Much faster than Yahoo, Faster than my corporate Notes client as well.
I like the label approach to filing things, rather than the folder concept.

Good work, Google.

Infopath and Sharepoint Portal Server

2004-03-31T21:46:00.000-05:00

I have been experimenting with using Infopath and Microsoft SharePoint Portal Server. It has been fun, and I think it can be a very useful combination. The main problem with using it in our environment is that we have yet to deploy Office 2003 Professional throughout our organization. I am working on that.

Simple form design is very straightforward. Drag-n-drop form controls - very easy. When you want to publish the form to the SPS server, that too is very easy. It creates a portal page for the form with a simple click to fill out a form interface. The portal page can show any field values from the saved forms.

One aspect that I still need to work on to get it working the way I would like is the merging of multiple forms in order to aggregate the data into a single view.

Another area that I need to work on more is scripting.

I still exist

2004-03-31T21:10:00.000-05:00

I haven't made any postings in a long time. Things just got too busy and to add to that, I am not sure anybody is reading this. All of my known readers were internal and got the things I needed to say directly.

Rich Internet Applications

2003-12-10T22:40:00.000-05:00

Two years ago we did a prototype of a rich client application using SVG. It was crude, but demonstrated the idea fairly well. We never did much with it. We got reorg'd a couple of times and has now been all but forgotten.

Macromedia has been flogging their rich client technology. I have finally gotten around to taken a look at what they have to offer. If you have 15 minutes to invest, to a look at their web presentation entitled: Macromedia - The business impact of Rich Internet Applications.

Tom, if you don't even have that much to spare, take a look at the section that talks about the etrade stock quote application. It is about 9 minutes in.

Consider their Watergate Hotel example. What would that kind of approach do to a banking application. Bill payment on one screen? (By the way, I checked out the applicaton on the hotel's web site... They don't use the new flash based application, it is the old one. Wonder why?

Unicode and Character Sets

2003-11-13T08:34:00.000-05:00

This article gives a very good overview on the topic of character encoding. Recommended reading for any and every programmer. Even if you only work in COBOL and IMS, there is a very good chance that your data will end up on a webpage.

A Pair of Interesting Articles

2003-11-10T19:19:00.000-05:00

This first articles talks about a new IBM server platform, the x455 which will let customers configure a server with between 4 and 16 processors in a manner which reminds me of lego. Need more processing power? Slide in another group of processors.

CRN : Daily Archives : IBM Pushes Scalable Itanium Server : 4:36 PM EST Mon., Nov. 10, 2003

But this second article could add a really interesting twist to the above capability. VMware's new capability provides a huge degree of abstraction from the underlying hardware. Workload can be moved around using a drop-and-drop interface.

InformationWeek > Virtual Servers > Multiplying Virtual Servers With New VMware Tools > November 10, 2003

So if you need more horespower on a server, you could add more x455 units, or move a virtual server image to another physical server.

Sweet.

IBM's case for VoIP

2003-11-08T09:55:00.000-05:00

Although a 240,000 user conversion to VoIP is a significant investment. IBM's existing Rolm based phone system is quite antique. It was pretty state-of-the-art at the time it was installed, but that is no longer true. By 2008 that will be overdue.

When examining the case for VoIP, it dosen't usually make sense to upgrade for the purpose of using the technology - there has to be some direct benefit to make it worthwhile. Otherwise you wait for your old equipment to age into obsolescence, or you acquire new premises.

SearchNetworking.com | Networking Decisions notebook: IBM's case for VoIP: "Johnny Barnes, IBM's vice president of global IT solutions and standards, told attendees that his company plans to migrate at least 80% of its more than 300,000 employees to voice over IP by 2008."

Acrobat and InfoPath

2003-10-03T12:48:00.000-04:00

Jon Udell has an interesting commentary onAcrobat and InfoPath. When InfoPath was first announced, under the name of xDocs, there was much hubub about it being an Acrobat killer. There certainly is some competition between the two. This will be an interesting fight once it gets started.

A Conversation with Jim Gray

2003-09-18T08:13:00.000-04:00

An excellent interview.

useit.com: Jakob Nielsen on Usability and Web Design

2003-09-18T07:25:00.000-04:00

I stumbled across this site on web usability. Something that gets left out of a lot of requirement specifications. Is usability a functional or non-functional requirement?

Perhaps they represent principles that drive out both requirement types.

Skype - VoIP Redux

2003-09-14T21:58:00.000-04:00

So, FreeWorldDial work well from home, but not behind the corporate firewall. Today on /. I came across this new application (Skype) that combines instant messaging with VoIP. It is what msn could be, if they didn't require pay-for-use subscriptions. Software install was fine, but will it work well with firewalls and VPNs? I'll give it a try and let you know.

Update on VoIP

2003-09-08T08:45:00.000-04:00

My original purpose for experimeting with VoIP was two-fold. 1) to talk to my daugther who just went off to college on the other side of the continent, and secondly to improve on my effectiveness while tele-commuting. The first objective looks like it will be a 'win' once she gets her high-speed (and always on) connectivity set-up.

The second objective was somewhat reliant on being able to tie into into the corporate VoIP phone network - over the VPN. This looks to be a challenge. It seems that so far, the best I will be able to achieve is Direct IP dialing. So does this put me any further ahead than using Sametime or Netmeeting? I think the answer is 'no'. To be practicable I would need to know the IP address of the recipient, that would likely be established via a sametime chat. It appears that I should focus on audio calls using sametime and getting everybody I know signed up for that.

Everyone Needs a Personal Server

2003-09-06T19:01:00.000-04:00

So I saw this on slashdot "Everyone Needs a Personal Server". I was kind of upset with myself. A long time ago - about when we started developing cell phone applications, I thought that this might be the way to go.

My thinking at the time was that rather than trying to jami everything into one device, spread them out a little. A palm-like device for display, an ear bud for sound, a little black box that would provide compute power and communications (I was thinking bluetooth at the time), and disk storage. So this is kind of like that.

I am sure I was the only one who thought of that too! ;-)

Experimenting with Voice Over IP (VoIP)

2003-09-03T15:42:00.000-04:00

Looking for ways to work better from remote locations, I have been exploring Voice Over IP solutions. So far I have signed up with Free World Dialup . I am phone number 57902 . It is a free service.

I have been using xten-lite as the softphone (SIP agent). It was very easy to install. The quality has been good so far with the biggest variable being the speakers and microphone used by the other end. Some feedback has occured with they are in close proximity.

The biggest limitation so far is that FreeWorldDialup is not useable from behind the corporate firewall - at least I haven't figured out a set of workable parameters as yet. Perhaps it won't be needed - that is if the new IP telephone system allows connectivity over the VPN from computer based softphones.

So if you want to play around, install a phone and sign up for FWD, and gve me a call - FWD# 57902.

GSM Security Cracked

2003-09-03T15:02:00.000-04:00

News item at Yahoo News says that Israeli researchers have cracked the GSM protocol and can listen in on GSM phone calls, and can also impersonate other callers. Iur first major concern would be "What about our HDML applications? Are our customers exposed?" I think the answer is "no", primarily becuase we would still be running TLS encryption on top of GSM.

Secure Web Sites versus keystroke loggers

2003-09-02T22:05:00.000-04:00

So how do you create a website that is safe from keystroke loggers and then like when the user may be using a public terminal and you can't secure the terminal. So far my bottom line conclusion is - you can't. (underlying assumption is that you don't wish to require millions of customers to buy special hardware like a security token). If any infiltrator can install a software keystroke logger, they could also install their own version of the browser, and with it any subversions they wish.

But what could be done to make things more difficult.
A virtual keyboard for entering a password (click keys with mouse)? This would help, unless the infiltrator is intercepting gui events and can dicipher what the mouse events correspond to. Also would be exposed to video camera over the shoulder.

One-time-passwords? Every time you logon you are provided with the password to use next time. Although that would work (mostly), it wouldn't be very popular - customers would forget.

Mouse based signatures. There is new research on this from the UK which suggests that mouse ballistics could work well as a 'signature'. Would have to devise a means to use it from a web application.

What do you think?