Kafka Streams has an edge over Service Fabric ?

Compared against the .NET/Azure offerings, the level of abstraction enabled by Kafka Streams for event processing while exploiting underlying Kafka message-topic-queue patterns is pretty neat. 
 
Did come across an interesting framework that used C# libraries over Kafka Streams by @tonysneed in GitHub too here : https://wp.me/pWU98-1v2
 
Hope Service Fabric Mesh Reliable Actor or similar offerings from Azure catches up with Kafka Streams in terms of seamless integration for distributed event processing.
 
For a start, assuring messages are processed 'exactly-once' is a basic requirement for most distributed systems. Yet to come across native frameworks in the .NET world that use Azure/Akka.NET streams/Service Fabric Mesh or the likes that enable essential distributed capabilities like 'exactly-once' and others with minimal developer effort :!

#azure #kafka #confluent #kafkastreams #eventsourcing #akka #distributedcomputing #cloudarchitecture

Software Engineering lost in the cloud?

It would seem the cloud is making you a lazy software engineer. Engineers these days are now have a ready answer for most of the architectural and design concerns - "its taken care at the cloud". This perception is scary and appears to makes any tom-dick-harry engineer with minimal to zero computer/software knowledge "become" "master" software-engineer overnight. This halo is bothering. Whatever happened to clean code / patterns essential to designing your software during the days of distributed computing setup in local clusters ? Perhaps none today cares about minimizing traffic across nodes and syncing time across nodes nor time sharing and optimizing resources during your minimal time at the node. Not sure the solution for this until you are choked to become yet another Harry. hashtag

Azure opens up for Private cloud - Windows Azure Platform Appliance

MS announced yesterday that Azure would soon be available for deploying on your local datacenters. Effectively, you could soon create private clouds using MS Azure. This is a great move especially if you were concerned about data security in the public cloud and subscription costs.

But still, the upfront cost could be high; this is not yet published. Check more here

This is also interesting since my last blog entry did refer to this thought!

Cloud Thoughts - 2

What if there was a mechanism to provision PaaS cloud environments (say MS Azure ) on public servers? Theoretically, if I had a free server(/cloud), I would install the "Azure runtimes" and add it to an existing cloud for others to use.

Registering a server to this free public cloud would then be a voluntary effort. This could have been a possibility if Azure allowed in-premise setup in the first place.

Can I call the end result as "Distributed Cloud Computing" ? Cloud computing that is distributed.

Cloud Thoughts - 1

A few cloud related thoughts


Cloud Cumulus

A single cloud access point that internally seamlessly talks with the subscribed cloud providers. Eg:- Subscriber A could subscribe to the services of Amazon, Google and Micrsoft (yes, Subscriber A is quite well off) cloud services. Subscriber A would deploy the same app on each of these servers and provide a single service URI. Subscriber A's customers would be serviced by one of the cloud provider seamlessly - perhaps with parts of the request being handled by more than one provider. Session states, data etc being shared across clouds is interesting.

For A's customer, there is only one cloud. The cloud of cloud providers / Cumulus Cloud accessed with a single entry point.

Perhaps, once the Unified Cloud Interface (UCI) is in place, this could be built?


Upgrade Ease

How easy is it to upgrade a cloud based app that is actively serving hundreds of users? Came across this for Azure :  http://msdn.microsoft.com/en-us/library/ee517254.aspx

Not sure what happens to the application state. If an 'In-Place' upgrade is followed, does it mean that at a particular point in time there could be two instance of the same application running on different versions?


Chess On Cloud

Though there are numerous instances of distributed chess engines, has anyone attempted to get a chess engine on the cloud? I guess the only person who can try this out today is Bill with his Azure and his $'s.

Azure Computing Usage, Metering etc and how MS gets richer

As per the MS release, the azure usage is to be calculated based on the 'compute instance'. $0.12 per hour it appears is for the 'small compute' instance size. For others, multiply this by the number of CPUs as per the below image :

Source ( http://www.microsoft.com/windowsazure/offers/popup.aspx?lang=en&locale=en-US&offer=MS-AZR-0003P )

Caveats

Based on your requirement, you could go for the small/medium/Large/Extra Large instance size. Things sound good until we dig a bit deeper on this. There are a couple of weird bits about the compute metering:

1.) If your application instance has 10 roles (worker/web...), each role would add up to the hrs!

Eg: - if you have 10 roles active simultaneously for 1 hr, you are charged for 10hrs.

2.) There is no part calculation. Even if you had you application active in Azure for 5mins and you delete it , you would end up paying charge for 1 hr!. This also is effective for the roles:

Eg: - if you have 10 roles active simultaneously for 5minutes, you are charged for 10hrs (yes, 10hrs)

3.) The compute usage is not determined by the actual computing/processing usage. Say even if your role/instance is idle for 50mins in the 1 hr, you are still charged for 1hr! This means that as soon as you have your application deployed, you get billed - immaterial of its computing usage.

Simplifying it down, your usage charge for a month = Number of roles in your instance * 24 (hrs in a day) * 31 (days in a month) * Subscription-factor.

Where, Subscription-factor = 1 for small, 2 for Medium,4 for Large , 8 for Extra large.

4.) Another interesting bit is about the alignment with the hours - it appears if you deploy your application at 3:50pm, you would have to pay an additional hours charge (2pm-3pm, 3pm-4pm).

5.) Are you a developer? There is no developer account/scheme wherein you could test out your application in Azure for free, yet.

Anything that adds to the MS revenue is good (for them).

Update 15.05.2010

For the various queries raised in this regard to the Azure support team, the answers from them haven't been direct, but rather confusing. Please do not rely on this post for deducing the cost, but contact the Azure team directly.

Quickly calculate Azure ROI/TCO

Interested in quickly calculating the ROI/TCO for your application once it is deployed in Azure? Check out these two:

1.) http://www.microsoft.com/windowsazure/tco/

2.) http://neudesic.cloudapp.net/azureroi.aspx

View/Query tables/data in Azure Dev Storage

When deploying application on the development fabric, you would usually need to actually view the dev storage - say check out the tables, write a couple of SQLs against it etc. OOB, there isn’t any support in VS2010/tools from MS. Note that development fabric is different from the Azure Storage in the cloud. Development fabric, dev storage resides on your local machine.

A very good tool you could use to access the dev storage for free (in addition to the azure store if you are a registered user) is Cloud Storage Studio from Cerebrata. Check more here:

http://www.cerebrata.com/Products/CloudStorageStudio/Default.aspx

Do let know if you come across any more free/thin/sleek/nifty tool that works.