Enterprise Architectures - skills, perspectives and insights - Tom Graves replies

Tom Graves has given deep insights (bit philosophical, but reality ) on building up skills on enterprise architecture from a professional and personal perspective; a query I had raised him earlier.

Read more here : http://weblog.tomgraves.org/index.php/2010/11/20/creating-a-career-in-enterprise-architecture/

HoC is up on codeplex

HoC - A distributed cache implementation using .NET 4.0 is now available for download with source at http://hoc.codeplex.com/

HoC = Herd Of Cache.

Azure opens up for Private cloud - Windows Azure Platform Appliance

MS announced yesterday that Azure would soon be available for deploying on your local datacenters. Effectively, you could soon create private clouds using MS Azure. This is a great move especially if you were concerned about data security in the public cloud and subscription costs.

But still, the upfront cost could be high; this is not yet published. Check more here

This is also interesting since my last blog entry did refer to this thought!

Cloud Thoughts - 2

What if there was a mechanism to provision PaaS cloud environments (say MS Azure ) on public servers? Theoretically, if I had a free server(/cloud), I would install the "Azure runtimes" and add it to an existing cloud for others to use.

Registering a server to this free public cloud would then be a voluntary effort. This could have been a possibility if Azure allowed in-premise setup in the first place.

Can I call the end result as "Distributed Cloud Computing" ? Cloud computing that is distributed.

Cloud Thoughts - 1

A few cloud related thoughts


Cloud Cumulus

A single cloud access point that internally seamlessly talks with the subscribed cloud providers. Eg:- Subscriber A could subscribe to the services of Amazon, Google and Micrsoft (yes, Subscriber A is quite well off) cloud services. Subscriber A would deploy the same app on each of these servers and provide a single service URI. Subscriber A's customers would be serviced by one of the cloud provider seamlessly - perhaps with parts of the request being handled by more than one provider. Session states, data etc being shared across clouds is interesting.

For A's customer, there is only one cloud. The cloud of cloud providers / Cumulus Cloud accessed with a single entry point.

Perhaps, once the Unified Cloud Interface (UCI) is in place, this could be built?


Upgrade Ease

How easy is it to upgrade a cloud based app that is actively serving hundreds of users? Came across this for Azure :  http://msdn.microsoft.com/en-us/library/ee517254.aspx

Not sure what happens to the application state. If an 'In-Place' upgrade is followed, does it mean that at a particular point in time there could be two instance of the same application running on different versions?


Chess On Cloud

Though there are numerous instances of distributed chess engines, has anyone attempted to get a chess engine on the cloud? I guess the only person who can try this out today is Bill with his Azure and his $'s.

Everyday Enterprise Architecture - The book - full download

It appears that Tom has put the full version of his new book for download for a limited period of time. Check out if it is still available here

Note: though the page might say its a preview edition, its actually the full book.

Review - Book - CLR via C# 3rd Edition by Jeffrey Richter

Having been a die hard fan of the second edition of CLR via C#, couldn't wait to get hold of the third edition. The third edition keeps up to the Jeffrey standard on covering from the basics to the internals of .NET 4.0 with C# 4.0. Starting with the rather interesting foreword by his wife, this book can be read end to end if you are already comfortable with .NET and have enough time to read through ~850 pages of pure core stuff.

There is so much of deep insight in most of the pages that it might take some time to absorb stuff. The best recommendation to attack this book is to read one chapter a day while preparing notes. Once you are done with the book, your notes together with the book can be one hell of a reference at any point. If there is only one book you read as a .NET developer, this is it. Even if you have been the best .NET developer of your league, each time you read this book there is something new learnt. This book covers a good wide area of .NET topics, sometimes delving deep too.

Sharing Richter's deep experience and know-how of .NET in plain non-geeky language has worked out well. Additionally, there are various tips and notes provided in the boxes and shaded text that are pure gems. Richter has made sure that the reader is thoroughly introduced to CLR concepts and in turn how things work out from the C# layer to the IL layer. Richter had made sure that most discussed basic stuff like hashcodes, interface v/s class, explicit v/s implicit interface definitions are discussed to a level such that no further questions arise. There are various instances where he has provided guidance on what to use and why, while sometimes providing historical background to these decisions.

There is good introduction to advanced topics like application domains in the context of hosting. With multicore and concurrency the talk of the town, new to 3rd edition are the 5 new chapters dedicated to discussion on threads and related classes in .NET. Discussion on TPL, PLINQ, spinning and locking etc are dealt quite thoroughly.

At the end of the book you would feel a better .NET developer, ready to take on your next development task in a more elegant manner - believe me.

Setting Physical Memory Limit for a .NET application

We needed to limit the physical memory used by a .NET application as it was taking too much of it and the other apps appeared to be in a struggle mode.

The straightforward way appeared to use Process.GetCurrentProcess().MaxWorkingSet. Strangely, no matter what we tried, this never worked. Yet to figure out why it does not work - if anyone has a clue, please ping.

Anyways, looking at options we came across job objects in Windows that allow setting the max working set. So the approach that finally worked was this:

1.) Create a Job object using CreateJobObject() Win32 call

2.) Setup the memory limits against this job using
SetInformationJobObject()

3.) Assign our process to this job using AssignProcessToJob().

Now, when we look at the task manager, the physical memory assigned to this process never goes over the specified limit. All good.

Note that this is applicable only for the physical memory and not for the virtual memory - no limits can be set for this [?]. The physical memory limit is affected when the application is paged-in from the page file into the memory.

Exploiting ObjectFromLresult() to get the IHTMLDocument2 from a window handle

Assume you have a window handle to a html control (which implements IHTMLDocument2) within a browser page (which is an IWebBrowser2), how would you access the HTML elements?

This is interesting as with just a window handle, it was relatively unknown on how that can be mapped to an object.

What you need to do is this:

1.) Register the windows message WM_HTML_GETOBJECT

2.) Send this message to the window handle we have in hand using a SendMessageTimeOut(). The out parameter in the lpdwResult returns you an UIntPtr to the object after the call.

3.) Next, use this out parameter as part of the ObjectFromLresult() call:
ObjectFromLresult(result from sendmessagetimeout, IHtmlDocument, 0)

4.) Cast the result from ObjectFromLresult() to IHTMLDocument2.

5.) Use the all property to get the HTML elements.

The powerfull function here definitely is the ObjectFromLResult() which effectively returns you an object that is 'accessible'. Accessible in terms of the user accessibility factor.

A framework worth checking out in this regard is the UIA framework that allows manipulating individual entities on the screen be it winforms, browser controls...

Azure Computing Usage, Metering etc and how MS gets richer

As per the MS release, the azure usage is to be calculated based on the 'compute instance'. $0.12 per hour it appears is for the 'small compute' instance size. For others, multiply this by the number of CPUs as per the below image :

Source ( http://www.microsoft.com/windowsazure/offers/popup.aspx?lang=en&locale=en-US&offer=MS-AZR-0003P )

Caveats

Based on your requirement, you could go for the small/medium/Large/Extra Large instance size. Things sound good until we dig a bit deeper on this. There are a couple of weird bits about the compute metering:

1.) If your application instance has 10 roles (worker/web...), each role would add up to the hrs!

Eg: - if you have 10 roles active simultaneously for 1 hr, you are charged for 10hrs.

2.) There is no part calculation. Even if you had you application active in Azure for 5mins and you delete it , you would end up paying charge for 1 hr!. This also is effective for the roles:

Eg: - if you have 10 roles active simultaneously for 5minutes, you are charged for 10hrs (yes, 10hrs)

3.) The compute usage is not determined by the actual computing/processing usage. Say even if your role/instance is idle for 50mins in the 1 hr, you are still charged for 1hr! This means that as soon as you have your application deployed, you get billed - immaterial of its computing usage.

Simplifying it down, your usage charge for a month = Number of roles in your instance * 24 (hrs in a day) * 31 (days in a month) * Subscription-factor.

Where, Subscription-factor = 1 for small, 2 for Medium,4 for Large , 8 for Extra large.

4.) Another interesting bit is about the alignment with the hours - it appears if you deploy your application at 3:50pm, you would have to pay an additional hours charge (2pm-3pm, 3pm-4pm).

5.) Are you a developer? There is no developer account/scheme wherein you could test out your application in Azure for free, yet.

Anything that adds to the MS revenue is good (for them).

Update 15.05.2010

For the various queries raised in this regard to the Azure support team, the answers from them haven't been direct, but rather confusing. Please do not rely on this post for deducing the cost, but contact the Azure team directly.

Quickly calculate Azure ROI/TCO

Interested in quickly calculating the ROI/TCO for your application once it is deployed in Azure? Check out these two:

1.) http://www.microsoft.com/windowsazure/tco/

2.) http://neudesic.cloudapp.net/azureroi.aspx

View/Query tables/data in Azure Dev Storage

When deploying application on the development fabric, you would usually need to actually view the dev storage - say check out the tables, write a couple of SQLs against it etc. OOB, there isn’t any support in VS2010/tools from MS. Note that development fabric is different from the Azure Storage in the cloud. Development fabric, dev storage resides on your local machine.

A very good tool you could use to access the dev storage for free (in addition to the azure store if you are a registered user) is Cloud Storage Studio from Cerebrata. Check more here:

http://www.cerebrata.com/Products/CloudStorageStudio/Default.aspx

Do let know if you come across any more free/thin/sleek/nifty tool that works.

Concurrency & .NET

With each versions of the windows programming libraries, the options for concurrent programming seems to be on the rise. Gone are the days when you had to start with a plain CreateThread() Win32 call (remember setting all those security attributes?). Then came the wrappers right from CThread in MFC, TThread in VCL (a more elegant version - Delphi ruled those days).

With earlier versions of .NET, you had the Thread class, the BackgroundWorker class and highly recommended ThreadQueue class. (lets not worry about all the sync objects that came along). With multi-core machines all around, the possibilities in .NET 4.0 are endless :

a.) Parallel Extensions (PLINQ + TPL)

Integrating parallelism right into the framework design while expoiting the extension methods has made expressing concurrency easier. Had a loop that you wanted to execute in parallel? Just use the Parallel.For().
1 core? 2 core? n core? Not sure how to exploit them? Just use the framework provided by TPL (Task Parallel Library) - your applications would scale (not worrying about the internal design/syncs for the moment) based on the number of the cores. Nice. The best part is, C# language and the supporting framework structure appears to move towards the functional programming paradigm - wherein you are not worried about how to do the job but more about what to do. LINQ, TPL, Parallel-extensions etc seems to be inspired by this functional paradigm as in Haskell [my current interest area)] / F#.

Want to dig real deep with some great samples ? Check this out : http://code.msdn.microsoft.com/ParExtSamples

b.) Axum

A very interesting .NET programming language from the MS research yard to check out. A language built with concurrency as the primary design objective. You have 'agent's (think about a block of code being executed independently like threads) talking with each other through the 'channel's using the 'message's (think about the all sync-objects you used to get two threads to talk with each other, but easier). Very promising - you could write your core domain objects in C#, use them within Axum wherein you would ave laid out your concurrency logic.

Check out http://msdn.microsoft.com/en-us/devlabs/dd795202.aspx , http://en.wikipedia.org/wiki/Axum_(programming_language)

c.) DirectCompute

Would like the exploit the massive processing power of your GPU? Check out the DirectCompute library. A DirectX 11/10 based framework that lets you offload tasks onto the GPU - awesome. In similar lines, also check out Brahma framework written by my ex-collegue Ananth at http://brahma.ananthonline.net

Dont miss the DirectCompute session video (http://microsoftpdc.com/Sessions/P09-16) which also showed some cool applications. Was amazing to see the computationally intensive job being done by the GPU while the CPU stayed at ~0% utilization !

d.) Dryad

Yet another product from the MS research aresenal, Dryad appears to be more targetted at making writing distributed applications easier. Need to check this out in detail - once I find an HPC server to do the installation, then perhaps port DES to it?

Check it out further at http://research.microsoft.com/en-us/projects/dryad/