Tuesday 13 October 2020

M&A and TOGAF

During an interesting discussion online on Mergers & Acquisitions, a basic question arose — if we consolidate technologies & tools used across the two merging companies, would it suffice most of the Architecture needs for the new company ?

Maybe; but in most cases, No.

A more formal approach is required to make sure we do not end up with a half cooked chowder served in a platinum goblet. We need means to formalize a recipe that takes care of most the business & stakeholder concerns while making sure we have added essential quantities of innovation and budget to the recipe.

What if we could exploit learning from TOGAF and its 4 domain pillars (BDAT) as the base line?

As the first essential requirement, management, stakeholders and technology leaders must define and agree upon an Architecture vision. The vision must represent the desired state of Architecture that cuts across the BDAT (Business, Data, Application and Technology) pillars. Furthermore, vision must act as the means to communicate with other partner leaders on where the new company is headed in the next 3–5 years.

Think of vision as a simple but appealing menu at the Michelin starred restaurant — — just enough to interest the diner. For typical small to medium enterprises going through a merge, think of at least 2–3 months to define this, as this would essentially become the guiding star for the rest of the architecture detailing exercise in the coming days.

Once we have defined the vision, it’s critical to have the current state of architectures ranked against the vision (think of maturity models). This could be as simple as 1,2,3,4,5 with the vision ranked at 5, while current architectures at rank 1; especially if we just adopt everything from both companies as-is before going through the below exercise.

Each cycle of the TOGAF ADM in the coming months should help us get to rank 5 as we reassess our rank periodically — every quarter/year. This is similar to Michelin Star 1 going until 3.

Following the TOGAF ADM is quite perfect for our need while detailing each of the BDAT pillars.

Business (Common processes which are procurement, operations …), Data (kind, tools, policies …), Application (Toolsets, policies) , Technology (Service Registry, SOA, micro services, neural networks…) pillars require many viewpoints to be created as required.

In addition, typical cross cutting viewpoints like Devops, Infra, HR, too must be assessed and detailed during the ADM.

Carrying ahead with the BDAT definition, ADM does provide means to define the governance model (who, how, what) and when/who can change/refine the governance model itself.

Now is the perfect opportunity to define the road map for the next couple of years for the merged company that also helps better the targeted architecture rank.

As we observe, ADM does force us into absorbing a formal mechanism to identify the perfect recipe for our new architecture. ADM compels us to look into opportunities (even across innovation programs active in the two companies) that could pop up during the merger that could further lead to defining new business processes/tools/use cases/products too.

Once the first cycle of the ADM is complete, we could have reference enterprise architectures that partner businesses can consider. All documentation including the reference models, process changes, view points, governance models, recipes�, principles could now be captured in the TOGAF enterprise continuum.

Soup is now being served. This time it was well cooked and served in a proper china soup bowl.

EventChain

Applying Blockchain to Event Sourcing

Event Sourcing pattern at the core requires an event store to maintain the events. What if we add these events as it arrives into a blockchain ? This should effectively make sure the events have not been tampered with. The plan would be to initiate typical blockchain mining after which the event is added to the “block-chain of events” — an “EventChain”.

The definite side effect is that until the mining is complete, the business transaction cannot be internally marked as complete. Considering the time typically taken for mining, this would probably be an offline job.

Tamper Proof

The typical challenges faced by organizations who employ event sourcing and the event store is about securing the events. What if the DB admin for the event store manages to inject/remove events ? The replayed events and resulting projections are no longer valid in this case. Event chains should solve this issue for typical event stores.

Exploit the distributed infrastructure.

For private event chains , where businesses do not want the chain nor events to be exposed, existing distributed systems/hosts can be exploited for mining. Your event store DB cluster hosts, event sourcing services hosts, API hosts, cache cluster hosts and others that are spread across geography could be exploited for the same.

GDPR Challenges

There are cases where regulations require personal data to be removed from all data stores. In our case, this is about removing the related set of events from the event chain. Without the event chain, removing events from the event store was quick and easy.

Resetting the event chain when events are required to be deleted is challenging especially if there have been many events after the event(s) in concern. This would require re-mining the rest of events after removing the event(s) that had personal data all the way down to the most recent event. As this is an extremely time and compute intensive operation, it’s not recommended to store events that contain personal data in the event chain.

Snapshots

As the events from the event store can be played back to recreate a state at a point in time (“projections”), we could in fact have “snapshots” to identify a specific projection in time. We could link this snapshot as a child branch to the main event chain tree such that it’s not required to recalculate the projections each time; while making sure the projections themselves have not been tampered with.

We could look at having many child branches/trees for the different filters/conditions too.

Monday 12 October 2020

Kafka Streams has an edge over Service Fabric ?

Compared against the .NET/Azure offerings, the level of abstraction enabled by Kafka Streams for event processing while exploiting underlying Kafka message-topic-queue patterns is pretty neat. 
 
Did come across an interesting framework that used C# libraries over Kafka Streams by @tonysneed in GitHub too here : https://wp.me/pWU98-1v2
 
Hope Service Fabric Mesh Reliable Actor or similar offerings from Azure catches up with Kafka Streams in terms of seamless integration for distributed event processing.
 
For a start, assuring messages are processed 'exactly-once' is a basic requirement for most distributed systems. Yet to come across native frameworks in the .NET world that use Azure/Akka.NET streams/Service Fabric Mesh or the likes that enable essential distributed capabilities like 'exactly-once' and others with minimal developer effort :!

#azure #kafka #confluent #kafkastreams #eventsourcing #akka #distributedcomputing #cloudarchitecture