Goliath and IT Security

Regardless of whether you hold any special significance to the Judeo-Christian scriptures or not, I suggest that there is wisdom to be gained from their study. A large number of topics are covered in these ancient texts, including IT Security.

Yes, IT Security. The story is told about a long running battle between the Philistines and the Israelites (1 Samuel 17 1-53). Everyday the Philistines would send out their champion, Goliath, and challenge the Israelites to a one-on-one battle. Winner takes all. The big problem was, and I do mean BIG, is that Goliath was a giant. One big dude. No Israelite solider knew how to fight such an enemy. So they cowered. The philistines laughed and were comfortable.

One day this kid comes along bringing lunch for his soldier brothers. His name was David. He is witness to Goliath's daily challenge and like most young idealistic people, who can solve all the world's problems, he says, 'yo bros, why aren't we doing something'! They laugh him off. But David persists and soon enough finds himself facing the giant; Armed with the tools he knows best. Goliath laughs and mocks him. David splits his skull with a rock from his slingshot. Fair trade. Goliath dead, the Israelites win and the Philistine army scurries away.

So how do I get from an ancient military battle to IT security? Goliath was proud, he thought he was invincible and so did the rest of his army. He likely was invincible against all attackers who engaged him in the defined model of combat: sword, shield, spear. David didn't play by the rules, he even tried on standard armour but quickly realized it was not going to work for him. Instead he thought outside the box and made use of tools that were not expected on the battle field and won.

All too often in the IT context I hear people say things like: 'our software is perfectly secure', 'It cannot be hacked', 'I signed to SOX attestation so I know that security cannot be circumvented'. Ya - and David can't kill Goliath. We all too often fail to recognize the difference between the truth regarding what is possible and our ability to conceive of what is possible. We only truly know the bounds of the possible once we have exceed them.

If you are convinced that something cannot be hacked, broken, violated, or circumvented then what you are truly admitting to is the limit of your imagination.  And never forget about the possibility of the $5 wrench.

Note taking applications.

I have had my iPad for about 6 months now, and it has certainly changed the way I use computers. It has become the one device that I use the most. Even more than my work laptop - but that has a lot to do with the amount of time I am spending in meetings these days.

In meetings I do use my iPad for note taking. The most common application I use for that is Evernote. I like that it automatically synchs my notes back to my computer. I have been taking monthly subscriptions to the premium service so I can have offline notebooks as well. The one short coming I find is that I cannot draw with Evernote. So I have started looking for another program to fill the gap.

I have been using Adobe Ideas for a bit as well. It is good. I like the zoom in and out capability. It also smooths out jittery lines nicely. But it does not work so well for written notes. The new version allows for emailing of completed drawings. I would like some better interop with Evernote.

With Evernote's Trunk feature there was an app called Inkest. I gave it a try. It seems to be geared towards artist who would like to sketch. I didn't find that it worked well for written notes. It does allow for drawings to be saved to Evernote's web site (not the local app). If you are connected that doesn't page much difference, but if you are not connected you will have to remember to Upload your notes latter.

On to my third try. InkShelf. It has a nice notebook interface similar to iBooks. Some templates that help get things organized. It does have the nicest writing experience ... The ink flows really nice. The multipage paradigm within a notebook is nice. It does allow you to pick different papers for your notebooks, which is nice, but it is a choice for the entire notebook, so if you want lined paper on one page and grid paper on the next, tough. It does allow you to save your notebooks to both Dropbox and Evernote.

There are two features from all of these that I am still looking for. The ability for the app to transcribe cursive writing into typed notes and the ability to edit the notes once they have been saved to a cloud mechanism.

I have used penultimate on other peoples iPads. It seems nice, but I think I prefer InkShelf.

The next thing I need to get is a stylus.

Free Music at Walmart

I just downloaded six free songs at walmart. Check it out.You'll need a walmart account.

Camera Connection Kit.

I have been giving the Camera Connection Kir for IPad a pretty food work while on the road.

So far I like the capabilities. But I do have some gripes. I am hard to please.

First, it works as expected. I am working with CF cards from my Canon. All shots are RAW. It can see the RAW images and import them, but I think it is just grabbing the embedded jpeg. I don't know for sure.

Things I would like to see improved. I'd like to be able to:
- know how much space I am using. There is an app for that but it should be native.

- create albums and place pictures in them.

- edit places.

- crop, straighten and adjust.

- import videos (avchd) - may support others, but not my cameras format.

Overall I'd like "Lightroom for iPad". That does all of this. It should work in concert with the regular Lightroom as well. For example, import embedded jpegs (or better) into iPad, classify, assign metadata, titles, create collections, etc. Then later import same images as raw from card into lightroom for the desktop and have lightroom for iPad provide an update feed.

There are apps for many of those features, but it would be nice to have it all in one app. "Lightroom of iPad"?

I also did some two image HDR processing on the ipad using TrueHDR. Very easy and good results. See here.

TrueHDR sample

Happy Canada Day ... The Canadian Bookstore Arrives

Previously I lamented about the lack of selection in the iBook store. It was just the free titles at that time. Today the Canadian content came online and there is a lot more to choose from. I still find the selection to be thin.

One thing I didn't notice before was alerts. You can set an alert on an author so you notified when a new book by that author becomes available. It would be nice if you could set an alert on a search phrase.

A feature I like about the amazon store is the wish list. Sometimes I am interested in something but don't want to buy it quite yet. Amazon let's me make a note about that by adding it to the wish list.

Zinio - I like it.

One of the apps I decided to try out based on somebody else's recommendation was Zinio. It is a magazine reader much like iBooks is book reader. There is a fairly wide variety of magazines available, but not as complete as I would like. They do offer a few complimentary issues which allows you to check out the experience without having to subscribe to anything. The app itself is free.

I browsed through the compliementary issues and was impressed enough that I have subscribed to a two. The text clarity is excellent, images are good as well. These magazines are more than just the print version render in electronic form. The ones I have explored have embedded videos, and slide shows. It is generally an engaging reading experience.

One quibble have will point out is the app is not the most stable. I have found it to crash about every 10 to 15 minutes. Long enough between crashes to be tolerated but definitely annoying. Fortunately it does a pretty good job of saving where you left off when you restart the application.

I have not done much to explore oth magazine readers. Have you? If you have found a good one leave a comment to let us know.

Thoughts on iBooks

Perhaps it is because my iTunes account is Canadian, but I have found the content a available on the iBooks store to be very disappointing. Close to useless for anything except old classics. I need to read some of them anyhow. Hands down the selection on Amazon is way better. However, I do think that iBooks is a better book reader than kindle. Oh well. The kindle read works quiet well, but iBooks just has a little extra polish.

What kind of polish? The bookmarks are nice as is the position within chapter and book. It seems less abstract than kindle. The page turn animation gets a lot of oohs and aaahs, but I am not so sure how much better it really makes the reading experience.

The one thing I really like is iBooks PDF feature. I have tons of PDFs; manuals, books, articles that I either want to read, or would like to have handy. A great example of a manual I have placed in ibooks is the manual for my camera.

Do you have a dropbox account? If you don't consider getting one. The drop box app makes it really easy to grab a PDF from drop box and then add it to iBooks. Slick. Get your own account here.[full disclosure - if you use that link I will benefit by getting more space. So please use it ;-)]

One big question remains - i mentioned it awhile ago when I wrote about the kobo. Just how many electronic bookshelves do you want to manage. I think I would prefer one shelf, connected to the amazon store which I could read with iBooks. Unlikely in the short term I think.

One improvement I would like to see to iBooks itself is a better way to manage a large collection of books.

Flash - my verdict

Much has been written about this by others so I'll just recap my experience. Are there websites I visit that use flash as an essential part of the site's purpose? Yes. Would I like to visit them on my iPad? Yes. So, if they want me to visit, they'd better change. I think that goes for my 20-something children as well, who want to borrow my iPad rather than hauling out their laptop.

Ironically of the the sites that I really wanting to use, bad enough to go to my windows machine is the Adobe store. It is all Flash based. (and not well designed either, but that is another story). I have been unable to complete my order there, I may buckle and actually call customer service, but only because I want the product sooner rather than later. If it had been available on amazon for immediate ship, it would have been bought already, and done from my iPad.

New family member - welcome iPad

I finally broke and bought an iPad. My rationale is that it does appear to be defining a new class of device and even though the next version will be much improved, I needed to understand the device now. For me that means using one as much as I can. Those who know me know that I will use it heavily for as much as I can.

I have had it now for a week; 32GB wifi. My rationale for the size was that it was bigger than my 16GB touch, and I could save a few dollars over the 64 model. Wifi only as I anticipated that if I really need mobile data access I would get a MiFi. I was also hoping that I would be able to pickup the hydro-one wifi signal from my desk. They offer an iPad data service for $5 per month. Unfortunately, I have not been able to make that work. Free open access does exist in many places downtown so generally am not too bad off. I think I will still miss the gps function.

I also purchased the dock, the VGA cable and the camera connection kit. The later because I hope to experiment with it as a tool for photographers as well.

So far the experience has been good. There was a good chance of it's new life with me when it was not available. Maybe I should have ordered two - or three. It has been very popular with the family.

That should be a good start. Of course this is all written on the iPad itself using the blogger web application. Works well.

The iPad and TV?

In this post, Dave wonders if the iPad will kill cable TV news. In my opinion they are at severe risk if they don't respond in some way.

I rarely watch TV to begin with, but when I do, I take my iPod Touch. It won't be too long before that becomes my iPad - unless my wife steals it.

My kids are more likely to watch tv on their laptops than on our tv.

My conclusion? Cable news out, fiber to the home in.

International Kindle not shipping until Oct 19th

See the Bloomberg article. http://www.bloomberg.com/apps/news?pid=20601087&sid=aiywhz5s9L7g
Given that info it is not surprising that Canadian orders are declined. Although Amazon should have thought about accepting pre-orders.

in reference to: Canada snubbed as Kindle goes global - The Globe and Mail (view on Google Sidewiki)

Sidewiki - kind of cool

Since I like recursion, I'll make a sidewiki entry about my first sidewiki usage. Of course I'll post that to my blog as well.

I wonder if they will tie this into Google Wave?

in reference to: Terry's Technology Topics: My first test of sidewiki (view on Google Sidewiki)

My first test of sidewiki

Not that this is a race, but I did need to beat Tom to the punch on sidewiki. Just because. It may be an interesting and useful tool. Lets see how it goes.

For this first test, I'll share the entry to my blog as well as Twitter.

in reference to: Google Sidewiki (view on Google Sidewiki)

Brilliant Attacks

Brilliant attacks. (especially the laser one !)
- Monitor the powerline and detect and decode keystrokes from 15m away
- Monitor vibrations on desktop object via laser reflections and decode keystrokes.

The more secure we think our systems are, the more we must remind ourselves that there are styles of attacks that we cannot conceive.

If anybody tells me ‘this is complete secure, it is unhackable’, then this I know for sure about the speaker is that they are NOT very imaginative. They might not be thinking very hard about possible attacks. To me this makes products from vendors who make such claims less secure, they will be blind-sided by some hacker with more imagination.

What is Scalability?

In computer systems we usually mean "linear scalability" and describes a relation between two measures. It is typically a reference to capacity. Two common measures are CPU capacity and number of users. If an application supports 10 users with 1 CPU and 20 users with 2 CPUs and so on until the numbers get quite large, we would say that it is scalable. because "number of CPUs" = 0.1 * "number of users".

It is a bit more complicated than that because we often have other constraints such as "user response time remains constant" and memory use scales as well. My first example also ignored the possibility of a constant offset, which would represent some fixed overhead. perhaps one-half of a CPU is required even if there are no users. For those who remember their math, y=mx+b is the equation which describes a line.

In my first paragraph i mentioned a possible exception "until the numbers get quite large". How large is large? That depends on the situation the solution is in. We typically would consider sizes that are significantly larger than what we expect, but still within limits. These limits can vary by the nature of the solution. If a workload is driven by internet stock trades, our degree of growth we could expect is much more volatile than if it is the number of Canadian branch locations for a bank (which is pretty much already saturated).

It would be false to assume that the linear scale continues without bound. Linear scale without bound, would be a truly rare situation. At various points as workload grows you will run into "walls". They are called that because when you look a the graph of this situation your resource usage grows much faster than your workload - as if it hits a wall.

These wall situations usually occur because some other resource becomes saturated. For example, perhaps your database server "maxes" out. In that case adding more CPUs to your application server won't help. But, perhaps re-engineering the database server will remove that constraint and allow for further growth. Sometimes these walls are "hard"; that is re-engineering won't alleviate the constraint.

There are two sources of these hard walls: coding and architecture. Some may argue that "coding" is just a different type of engineering constraint. I won't argue that, but in my company we have an engineering department that specializes in server sizing and configuration, and development departments that do the coding. so we classify them as two distinct problem types. I have another reason for that differentiation as well. Engineering constraints can usually be quickly fixed with the addition of more resources (server, CPU, memory, etc) or reconfiguration/reallocation of existing resources (add more threads, connections, heap). Ccoding problems on the other hand usually take much longer to diagnose, recode the problem area, retest and redeploy. A trivial example of this would be the replacement of a linear search with a hash table lookup.

Architectural constraints are more fundamental design decisions which can not easily be altered. For example a design decision that requires an application to execute completely within a single server. This might be a simple design that performs well - as long as you can buy a larger server. whether this is a good decision or not depends greatly on how reasonable your assumptions about the potential for growth may be.

Is scalability always a good thing? Perhaps not. it depends on what you are measuring. I recently read a product evaluation that said (incorrectly) that the product's license model was not scalable. The truth is that it is high scalable. The more of the product we used, the more we paid. It wasn't linear though, because volume discounts meant that unit costs dropped as volume increased (and that is a good thing!). What the author really meant was that they wanted NON-scalable pricing; they wanted a price ceiling (somewhat like a wall except on the other dimension). At a certain point of volume growth, they didn't want to pay anymore. a desirable feature for the buyer, bit maybe not the seller.

There is much more that could be written, horizontal versus vertical scaling. 'knees in the curve', etc. But until then you might like to read the wikipedia article.

8,000 US Banks

"the U.S. still has over 8,000 banking companies": Anybody who wants to understand the US banking industry need to understand this point. The referenced article gives just one viewpoint on the degree of diversity that is 'The US Banking Market'.

Wacky pricing... paper, ebook, and audiobook

Something is just plain wrong here. I have been listening to Taleb's "The Black Swan: The Impact of the Highly Improbable". A very interesting book, by the way. Recommended.

Anyhow, this is about price. I bought the audiobook from Audible.com so I could listen to it during transit times, etc. I paid C$15 for it. Expensive for a download I thought, but Amazon wanted US$21 for their audio download. The audio CD? US$26 !!

I decided to check out what a paper copy of the book would cost. I wasn't surprised to find it on amazon.ca for more money, but not too bad at $20 for the paperback. But I don't have any more space on my bookshelf.

So I looked for an ebook. I was shocked to find the price to be US$27 at several sites. That is insane. Finally, what about amazon/kindle? US$12. Much more reasonable, but you have to buy a Kindle. Sony's price is the same.

I think there is a bit of room here for price competition. I can't see any reason why the electronic download should not be universally half-price compared to the original media. This is somewhat the way it is for music. Perhaps libraries provide price competition on the paper?

One other thing I noticed. It doesn't seem to be general practice for a retailer to offer multiple media formats for an item - the one exception being the Kindle view on amazaon. Surprisingly the reverse was not the case. The regular amazon entry for the hardcover book did not reveal the other options available. A missed opportunity.

Prototype, Proof-of-Concept, and Pilot. Oh my!

These terms are often used in IT contexts often without much consideration to nuances. Whatever you call it, the first important thing is that you understand and state your objective. The second is that you meet it.

My suggested definitions.

Pilot: This is an implementation of a system that is often functional complete. It is typically deployed in production, but usually constrained to a small number of users. Although we hope everything is perfect, there is an expectation that there will be faults that require rework - otherwise we could have just gone full production. The fault maybe in deployment, code, design, or in user experience. A pilot is typically time-boxed. A pilot is usually fully productionalized from an operational perspective.

Beta: Similar in many aspects to Pilot. There is a lesser expectation that it is functionally complete, but it typically is. With a Beta, there is a much more explicit understanding that it is not final. It will change in the final release. A Beta is often supported, if at all, by a different organization than a production instance, typically developers. Traditionally a beta was not to be used for production, however some companies are making it part of their normal process - the never ending beta. Something that Google has done many times.

Release Candidate: A software build and might be view as living in the middle between Beta and Pilot. A release candidate may be promoted to pilot or production.

Proof of Concept: I view this as a very narrow and well defined activity. There is a well defined concept. The objective of the activity is to prove that the concept is viable in some aspect. Functionally it is only complete enough to meet the objective. The resulting code is not intended to be used for anything else - although most programmers will harvest some aspects for other things. Agile methods talk of 'early pain'. Significant projects has technology aspects that well be challenging and be a source of risk. It is desirable to execute on those aspects first, if there is a problem you want to know about it early so that you can change your plans or maybe cut your losses. This is what a proof of concept is about; if you are going to fail, fail early. Many times I have seen people propose a 'proof of concept', with no idea what concept they wish to prove; often what they mean is that they want to start coding. PoCs are not implemented in production.

Prototype: Perhaps this one has the most varied definitions: Experimental Prototype, Engineering Prototype, etc. In software development, a prototype is a rudimentary working model of a product or information system, usually built for demonstration purposes or as part of the development process. As part of an SDLC approach, a simple version of the system is built, tested, and then reworked iteratively until ready for use. Prototypes are not usually implemented in production. Go read the wikipedia article http://en.wikipedia.org/wiki/Prototype

Do you agree? Are there any characteristics of any of these terms that you think would help define them.

Unstructured, Semi-Structured, and Structured Data

I originally wrote this over two years ago and have intended to post it ever since. Not too late.

Often in the IT world we hear or even use these terms. But what do they really mean? Here is my view.

All bits and bytes that we deal with in the IT world we considered to be data (at least). It all has some form of syntax and structure, so what do we really mean, and why is it useful to distinguish between them?

These three classifications represent a continuum which spans from unstructured to structured data that represents the degree to which the data's semantic model (meaning) matches our processing requirements. In general what we are trying to describe is the readiness of the data to be processed in a particular business context .

For example, if the data in question is the raw audio recordings from the call centre, and the business context is we need to review all verbal instructions spoken by customer "Joe Smith" last year over the phone, we may consider that recordings to be unstructured. We have no easy way to process the request.

If we have augmented those recordings with additional data from other systems and have added customer number and call timestamp to the recordings (or an index) then we would consider that to be semi-structured data. Although we could quickly sift through the millions of minutes of recordings to get Joe's subset, somebody would still have to listen to the recordings to find the things that Joe said.

The structured data, in this case, could be represented by the actual transaction records that the call centre agent created in response to Joe's instructions.

Likewise, A TIFF image might be considered structured data within the Context of a GIS application (geographic information system), but might be considered unstructured within a mortgage appraisal application. (Perhaps even GIS would consider it unstructured since they might ideally wish to run queries over an image set to find all lakes larger than a certain size. That would be hard on untagged TIFF images.)

All the data we typically deal with has a known syntax, even if that syntax is only really understood by MS Word. And although a Word document may have semantic meaning to a human, that semantic meaning is not easily extract by computer. We consider a Word document to be unstructured (in most cases).

An Excel spreadsheet may have a well defined layout of rows and columns. Although Excel may find it easy to 'understand' its content, other programs may or may not. If they layout is regular and complete, programs other than Excel maybe able to extract that data from the spreadsheet and do useful things with it. We would consider that to be semi-structured. I suggest that the 'semi' aspect of the term introduces the concept of a degree of uncertainty. Perhaps this is because it source is not well controlled and the form (layout) may change and it suddenly becomes unstructured in our context.

Structured data has an aspect of surety about it . We know that there are 'fields', we know where they are, we know what values to expect. We know how to understand it. We expect there to be some kind of formal model which defines this structure, and we expect that there will be controls in place that enforce our expectations. We may often visualise such data as being a relational form stored in a RDBMS. But that is not a requirement.

All that said, here are my definitions:

Unstructured Data: Data which does not have the appropriate semantic structure which allows for computer processing within a particular business context .

Semi-structured Data: Data which has some form of semantic structure which would allow for a degree of computer process within a particular business context, but may need some human assistance. It may apply some heuristics but the process may fail due to volatility of the structure or incorrect assumptions about the structure.

Structured Data: Data which is well positioned to be reliably processed by computer within a particular business context. It has a well-defined and rigorously controlled syntactic and semantic structure. The elements of the data have a well defined datatype and rules about valid values and ranges. The meaning of these data elements is well understood in isolation as well and their relationships to other elements. Elements are also traceable to their originating sources and that path is verifiable.

Synovus to merge Americus and Albany banks - Atlanta Business Chronicle:

Synovus to merge Americus and Albany banks - Atlanta Business Chronicle:: Something doesn't seem right here. Consider these two quotes.

"Once completed, the combined banks will have $649 million in total assets in seven locations, Columbus-based Synovus said."

"On Dec. 19, Synovus received $967.8 million in funding from the federal government’s Treasury Troubled Asset Relief Program (TARP)."

They have received more in TARP funding than they have in assets!