Posts Tagged ‘Architecture’

Have you ever wondered how major sites are logically and physically built?  Not as some high level with shaded boxes but at real platform level details.

I’ve been
looking for a site like this for a while.
No theory here, just real details about the servers, platforms, db’s,
caching, apache mods etc. that the “big boys” use – Facebook, Digg,
YouTube, craigslist, MySpace, Wikipedia, PlentyOfFish, Amazon and more…

I was reading an old post by Simon Guest about removing the intimacy to create services and thought he had a great analogy comparing real life to SOA solutions.  I love these types of posts because they provide concrete examples which explain system interactions in a straightforward manner.  Gregor Hohpe posted something similar to this in Starbucks Does Not Use Two-Phase Commit post.  It’s a great read. 

I would like to take Simon’s post a few steps further.  Continuing on his example, let’s add a few more features such as:

1.  Message Correlation
2.  Message Expiration
3.  Concurrent Orchestration
4.  Policy-based security

When person A approaches person B to borrow $5, B must first make sure that he (1) knows who A is and (2) can trust A with this transaction.  Even though A looks like who he says he is, and he even knows B’s name and address, B has a strict “policy” that A must provide a state-issued ID (federated security) which B scans into his security manager to verify A’s identity. 

After his identity has been verified, B looks up A’s record and current outstanding balance.  B will check if his business rule stating that no one person can have an outstanding balance of more than $100 has been violated (business rules engine).  A promises to return B’s money in 7 days and that’s fine with B’s business rule. 

In B’s orchestration engine (his brain), he creates a new long running transaction where he listens, waiting for A to return his money, on or before today + 7 days.  He creates an alert in his process manager (his Blackberry) to send himself and person A an alert if he has not received the “money returned” message by that day.  If no message comes in by this time, a new process will be instantiated to mitigate the situation (the sendTonyToBreakLegs process) which receives a violatingEntityIdentifier message as its input message.

B is a very generous guy and let’s a lot of people borrow money.  He may have several long running transactions going on at any given time. In fact, person A could be going through some hard times and may have borrowed money several days in a row.  Each transaction is assigned a unique value so that when A starts paying back, he can start applying it the one closest to expiring.

When you look at daily life, it’s really one big integration project.  Orchestration, coordination, services, long-running transactions and asynchronous operations are all around us.  How well individuals deal with it is largely dependent on individual’s internal processing engines as well as accessibility to tools that specialize in the management of these processes (Blackberry, Outlook etc).

After reading Jesus’ blog post about ESB, I got to thinking.  At the end of the day, its really about just getting disparate systems to play nice together.  We can’t get countries in our own world to play nice together.  If we can agree that everyone in the world speaks a common language, it doesn’t mean that we’re going to play nice together.  Some will need a Translator, or at least a Broker to share the same idea, but with different implementations (think English-to-Spanish translator vs. German-to-Spanish).  The trick is to see and embrace those differences, encapsulate the difference, and figure out a way to mitigate them as quickly and painlessly as possible.  If you think a merger will ever be as simple as changing a configuration setting, creating a new Receive Location, and adding a Send Port, you’ve never been involved in a merger (and you’re using BizTalk).

To think there’s a standard to anything is a naive approach.  And to imagine that those standards will survive the test of time is even sillier.  Face it.  In some development environments waterfall actually works and agile really fails.  Its about the culture, skill set, management and so many other factors you can’t even begin to imagine.  Its about adaptability.

I always like to look to the business to solve our enterprise and technology problems (Eeeeh?).  If you can get to the people closest to the business, and they describe a customer as X with Y accounts and an account as N with M funds, then you are getting the sense of the business/IT model.  Everything else is just plumbing.  We love, as technologist, to find relations among everything, but we have to be forward enough to see that if the business says “we’ll never have more than 100 accounts for each customer” we make a decision to not enforce it in our entity models.  You can enforce it somewhere else, but make it configurable ;-)

I think an ESB is a great step in the right direction for many companies facing an ”IT identity crisis”.  If not for anything, then just because it forces anyone adopting it to take a step-back and look at their business domains.  It forces us to establish a canonical model of major business entities which I believe this is vital to the success of a business and IT strategy.  It seems people ascribe the word “canonical” to mean “never-changing, permanent, inflexible”, but this is a huge misnomer. 

That being said, if you want to change what your company or industry considers the canonical model of what a “sales order” looks like, you’d better have a darn good reason.  That reason needs to be signed off on by X, Y, and Z and it needs to be versioned and documented and syndicated to all other subsidiaries.  It shouldn’t be easy and you should need buy-in from several people making much larger paychecks than you. 

If you’re not in a position to create a canonical model of what a certain business entity looks like in your company, please don’t bother to play with an ESB.  It will only complicate your life when you find out that a sales order line item really doesn’t do a “real-life” sales order line item justice.  I suggest as a first step to step away from the technology and look at (and understand) your business domain.  This is the key to a successful business / IT strategy and its the promise of SOA deployed properly.  It’s all about business driven development.  Did I just coin that?  Apprarently not.

I’ve been involved in recent discussions on the divide between architecture
and technology.  I think many developers often confuse an
implementation technology with architecture.  I believe there is an
important distinction between the two.  This is often exemplified in
some people’s definition of SOA as “web services”. 
Apparently, if you can code a web service, you are officially a service
oriented architecture guru.  I was reading an e-book I found on MSDN’s
Architecture site and thought it provided a good metaphor depicting how
important it is to have an architecture blueprint and know the
difference between architecture vs. technology.  At the risk of
duplicated content on the web, here it is and the book can be
downloaded here:

Winchester Mystery House is an intriguing tourist attraction in the USA
near San Jose, CA. The Winchester Mystery House was the home to the
heiress of the Winchester fortune (amassed from the sales of Winchester
rifles). According to the legend, the heiress went to see a fortune
teller and learned she was cursed to be haunted by the spirits of
everyone ever killed by a Winchester rifle. The only way to avoid the
curse was to build a mansion – as long as she kept building the spirits
would leave her alone. She promptly hired 147 builders (and 0
architects), all of whom began working on the mansion simultaneously.
The builders worked on the mansion until the heiress passed away, 38
years later. The result of their efforts is a classic example of
implementation without architecture:

  • The mansion contains 160 rooms, 40 bedrooms, 6 kitchens, 2 basements and 950 doors
  • Of
    there 950 doors, 65 of them open to blank walls, 13 staircases were
    built and abandoned and 24 skylights were installed into various
  • No architectural blueprint for the mansion was ever created.

architecture with implementation generates chaotic and unpredictable
results – much like the Winchester Mystery House. Articles that try to
explain SOA and jump into a tutorial for building Web Services are
providing guidance for coding, not architecture. This is one of the
many reasons that SOA is so misunderstood today – the rush to promote
loosely coupled architectures focuses on the trees instead of the