Modern Enterprise integration Strategies

Editor's Notes

• #14: Messaging: Inbound, Outbound, Durable Messaging, Message Security, Message Assembly/Disassembly, Pluggable Adapters, Message Security, Message Assembly/Disassembly, Pluggable Adapters, Enterprise Single Sign-On, Service Enablement • Web Services based messaging (WS* standards)Windows® Communication Foundation (WCF) Adapters • WCF-BasicHttp • WCF-WsHttp • WCF-NetTcp • WCF-NetMsmq Orchestration: Flow Control, Port Bindings, Scopes, Design Patterns • Asynchronous Response Correlation • Aggregator • Composed Message Processor • Dynamic Router • Parallel Convoy • Scatter and Gather • Sequential Convoy • Splitter Business to Business Integration: Trading Partner Management, EDI Support, AS2 Support, Industry Accelerators • SWIFT • HL7 • RosettaNet Business Rule Framework: Business Rule Engine, Business Rule Authoring Business Activity Monitoring: Observation Model Design, Event Observation Engine, Reporting Management & Operations: Application Management, Tracking Configuration & Reporting, Host Management, Application Deployment, Best Practices Analysis, MOM Monitoring, Data Management AppFabric Connect Integration: Line of Business Applications (LOB) activities (Consume LOB operations as Windows Workflow Foundation (WF) activities), Mapper Activity (Transformation as a WF activity) RFID Platform Tools • Schema Editor • Flat File Schema Wizard • Mapper • Pipeline Designer • Orchestration Designer • WCF Service Publishing Wizard • Administration Console • MOM Management Pack • Best Practices Analyzer • Business Rule Composer • Policy Deployment Wizard • Orchestration Designer for Business Analysts • Observation Model Designer • Tracking Profile Editor • BAM Portal • BAM Manager • EDI User Friendly Schema Viewer • EDI Document Instance Generation/Validation • Trading Partner Portal • RFID Manager • RFID Tray • RFID Event Simulator • RFID Client Console
• #15: Neste cenário, um aplicativo de inventário, talvez a correr num mainframe, determina que o stock de um item é baixa e emite um pedido para pedir mais desse item. Ocorrem as seguintes etapas: The request is sent to a BizTalk Server 2010 application The BizTalk application requests a purchase order (PO) from the organization’s Enterprise Resource Planning (ERP) application. The ERP application, which might be running on a UNIX system, sends back the requested PO. The BizTalk application informs a fulfillment application, built on the Microsoft .NET Framework, that the item should be ordered Neste exemplo, cada aplicação comunica por meio de um protocolo diferente, utilizando os formatos de mensagens específicas de cada aplicação. Isto significa que o mecanismo de mensagens do BizTalk deve ser capaz de se comunicar com cada aplicativo no seu protocolo de comunicação e formato nativo e também converter as mensagens para o protocolo e formato exigido pelos demais sistemas.
• #16: The two main services in BizTalk Server 2010, the messaging engine and the orchestration engine O objetivo do BizTalk é processar mensagens. Toda a comunicação dentro e entre um servidor de BizTalk são baseados na troca de mensagens. Por esse motivo, o mecanismo de mensagens é essencial para todas as operações do BizTalk. The BizTalk messaging engine performs the following tasks: Receives inbound messages Parses inbound documents to identify their specific formats Extracts key identifiers and identifies applicable routing rules Delivers documents to their respective destinations including ports or orchestrations Tracks documents Message database The MessageBox database is a Microsoft SQL Server™ database that is used by BizTalk to store and route messages to orchestrations and send ports. When a message arrives in the MessageBox database, the metadata associated with the message is matched and evaluated to determine the services that subscribe to messages of this type. Publish-Subscribe Model BizTalk Server implements a publish-subscribe model for the routing of messages. In the publish-subscribe model, message providers (publishers) submit messages to a central store (the MessageBox), where subscribers (send ports and orchestrations) can subscribe to specific messages. After a message of interest is received by the MessageBox, it is sent to all subscribers
• #41: Sources: -- Where the message comes from -- Every bridge has one HTTP source -- Multiple sources can be configured for a bridge endpoint Destinations: -- Where the message is sent -- Determined by the routing filters Pipelines: -- Pipeline template plus configuration -- Template composed of stages; stages composed of activities -- Stages and activities execute on the request path, as well as response path
• #54: In the beginning there were apps. Actually, before the beginning there were digital computers, and then mass storage to store system state And pretty soon people wanted to set out formally codified rules that modified the state in formal ways. Data and logic. We used to call these “two tiered systems”. Nobody says that any more. What was the first system you worked on Greg? …. I did a bunch of numerical programming in the engineering program I was part of. And also I was a hobbyist so I wrote code just for fun. But the first job I got paid for was a two-tiered app. It was in my college years, a dBase II project. dBase was a sort of integrated environment with the data and logic all melded together. It was a order and customer management system for an investment brokerage company. I paid a lot of bills working on that system. A ton of people did. The world ran on these kinds of systems.
• #55: Applications were useful, so they proliferated. And it was good. But then the problem became interconnecting these things. The system that handled loan origination needed to integrate with the system that was used for business planning, or portfolio management. And that’s a problem we’ve never really 100% solved. That problem continues to this day, just on different scales. About loan origination - I just talked with a company that has a twelve year old system that does loan origination. They’ve run their business on this system for more than a decade. Custom built. They also built a portfolio management system – once the borrower signs for the loan, it becomes part of the portfolio. It needs to be managed, checked, verified. But: The two systems don’t share a common customer model. Also - There’s no feedback mechanism in the portfolio management back to origination. If there are patterns, if for example loans made for mini-golf places are more risky than loans made for car washes, there’s no way to feed that information into the risk rating during the origination process. This is typical. Happens everywhere. Even in “modern” enterprises.
• #56: There were a number of different approaches taken for interconnecting different systems. Numerous. But in 2002 to 2005, we all settled on SOA. What’s your view on SOA, Greg? Is it real? Useful? ….(basically, yes. It got overhyped and over-productized. But it works as a model)…. SOA is an approach for modeling large systems as services.   In contrast to objects. Anyone remember CORBA? DCOM? Those were distributed object models. Very popular. Objects were everywhere. OOPSLA was born. With the distributed objects model, cooperating applications were remote objects, and programmers could interconnect by invoking methods on remote objects. SOA’s different. Rather than invoking methods on objects, SOA said these remote things (applications) were really services, and we would send them messages. This was a profound switch. SOA’s Only Architecture
• #57: At that point, we modeled each standalone system as a service. And each service accepts one or more different sorts of messages. The list of messages a service could receive could be called the “contract” – and it was rigidly specified in the form of WSDL. We, as an industry, spent a lot of time focusing on the messages, describing them formally, describing how to map in-memory representations to on-the-wire, textual representations of messages. Trying to be very thorough. XML Schema. XML Data Binding. JAXB. XML namespaces. The effort we as an industry put into figuring out how to handle these messages. An astounding amount of effort. It amazes me. The XML Infoset. An attribute on an XML element can be namespace qualified. Remember that? A namespace with a trailing slash is different than a namespace with no trailing slash. Namespaces look like URLs but aren’t really. ----- “Services” can also be clients of other services. Extending that model we have a connected web of services. They all interoperate by sending and receiving messages according to the defined contracts. It was all about re-use of systems, with tight control. Primarily system-to-system integration. And while all of this was happening, the web was maturing.
• #58: The term “Bus” is an abstraction, an analogy. It refers to the hardware bus – the bus that moves data between CPUs and memory. But it can be more than an abstraction. It can be concrete. Companies deploy ESB products, to facilitate the control and management of the SOA interconnections. Logging, controlling the access.
• #60: The price of enterprise network has plunged steadily and continues to decline. It used to be expensive to acquire and provision a 10mb ethernet switch. Now 1gb ethernet is yesterday, and 10gbe is standard. Network is plentiful. Compute has followed a similar trajectory. It continues to drop in cost. Storage likewise. There was a time when people actually had to “reserve time” on a shared computer. Now, compute power is cheap and getting cheaper. And at the same time, the iphone Here’s the thing: SOA was designed with the old constraints in mind. The formality of SOA messages was necessary, because compute was still scarce, and network was relatively scarce or expensive. Proper system architecture had to consider the cost and availability of these resources, which meant we needed asynchronous queues in front of systems, so they would not be overloaded. This implied that the messages on those queues needed a great deal of formality, so they could be stored with all their context, indefinitely. This way, when re-constituted from the queue, everything was still there. Time of origination. Identity of the sender. Identities of other intermediaries. Response path. Many people shied away from using HTTP as a transport inside corp networks for system-to-system connection, because of the overhead of an http request, as compared to TCP. Have you ever heard of :”Chargeback”? There was a time when shared resources on a corporate network were metered, and if your system used those resources, your department was charged for that usage. Like a toll road. But we don’t have those constraints now. Now most enterprises have so much network capacity, it makes no sense within a corporation to charge for it. It’s like air. It also makes no sense to reject HTTP for its “overhead” in most casts. And there is less imperative to introduce asynchrony. Compute power is so inexpensive that most requests can be fulfilled RIGHT NOW, and systems can connect to each other directly. It’s more efficient this way.
• #61: One more huge change: Mobile computing.
• #63: APIs address a set of requirements that are different than those addressed by SOA. If you think of iterations in IT architecture, Some might say SOA was a necessary step for us to take. At this point though, we’ve got requirements being driven by the ubiquity of mobile computers, and the opportunity to be accessible in new ways to people who carry those computers with them at all times of the day and night. SOA wasn’t designed for that. There are a bunch of hard feature/functions that come from that. Rate limiting Oauth Auditing Analytics Quota management Versioning Developer enablement and onboarding
• #64: Design Metaphors are just ways of thinking about modeling a problem. For a while we used Objects as a metaphor. EJBs –distributed objects SOA – services Now, APIs are the metaphor we use. It’s THAT important. The metaphor gets us to think differently about framing problems. It’s not just lip service. Thinking differently guides the solutions we build. What’s different? The Focus on Programmability. It’s in the NAME, fer pete’s sake. The loose contracts. They’re described in human documentation form, or they’re not described at all. JSON for universal access. A simple data format. YAGNI. Loose contracts means things can change often, and they do! Change leads us to iterations, which begets agility. Change of the interface is simply not something SOA was aimed at. In fact the mantra for SOA was was: change the implementation, and keep the interface exactly the same.