Introduction to Messaging 3.7
- 2. Introduction to Messaging Goals • Learn how to configure Neuron ESB’s hierarchical topic based pub/sub messaging system • Learn the basics of what a “Message” is within a Neuron ESB Solution, • Learn how messaging can be used to decouple and abstract elements of a solution • Where and when it’s appropriate to use messaging and gain an understanding of its meta data model
- 3. Introduction to Messaging Lesson Plan • Introduction to the Neuron ESB Message • Messaging • Topics • Parties • Conditions • Subscriptions • Topic Taxonomy
- 4. A Neuron ESB Message is an instance of the Neuron.ESB.EsbMessage class. • Message Body • XML • JSON • Binary • Text • Header Properties • Neuron • SOAP • HTTP • Custom • Schema-less Introduction to Messaging { “OrderId”:”123456”, “Quantity”:”100”, “ProductName”:””, “CustomerName”:”John Smith”, “CustomerEmail”:”jsmith@live.com” } Http.StatusCode: 200 Http.StatusDescription: OK Http.Method:POST Topic: Online.Orders Semantic: Multicast MessageId: d8457917-da73-41c9-97ce-1f3… My.MyCustomProperty: Some custom data Neuron ESB Message
- 5. They are declared with a prefix and a name • Adapters use them to drive configuration at runtime • Services use them to drive configuration at runtime • Process Steps use them to drive configuration at runtime • Live with the lifetime of the ESBMessage • Can be preserved on Request/Response calls • Can be used in “Message Patterns” and “Conditions” • Can be set using the “Set Property” Process Step • Can be created and retrieved in a C# Process Step • Developers can use to drive custom processing • Developers can use to implement custom tracking • Neuron Auditing stores custom properties Introduction to Messaging { “OrderId”:”123456”, “Quantity”:”100”, “ProductName”:””, “CustomerName”:”John Smith”, “CustomerEmail”:”jsmith@live.com” } Http.StatusCode: 200 Http.StatusDescription: OK Http.Method:POST Topic: Online.Orders Semantic: Multicast MessageId: d8457917-da73-41c9-97ce-1f3… My.MyCustomProperty: Some custom data Custom Properties
- 6. • Only parties (publishers and subscribers) can communicate with topics • Communication with topics is enabled through subscriptions Introduction to Messaging Messaging Communication Neuron ESB Messaging Party Party Topic Topic Topic
- 7. Introduction to Messaging • Endpoints can host publishers and subscribers • Can receive messages on topics or directly in business process and workflow definitions Messaging Relationship to Endpoints Neuron ESB Workflow Runtime Workflow Endpoint Neuron ESB Messaging Neuron ESB Runtime Service Endpoints Adapter Endpoints
- 8. Introduction to Messaging There are 4 main entities in the messaging Publish/Subscribe section • Topics • Publishers • Subscribers • Conditions Messaging Entities Publisher Subscriber Subscription Subscription Conditions Topics Conditions
- 9. Introduction to Messaging Message Exchange Patterns Request/Response One Way Direct Async/Broadcast/1-Many Message Exchange Patterns Distributed MOM Centralized Management Foundation for EAI Platform Enables Service Broker Benefits
- 10. Messaging : Demo • Purpose: • Familiarize users with the messaging section of the Neuron ESB Explorer. • Objectives: • Acquaint users with the following aspects of the Messaging section • Topics • Publishers • Subscribers
- 11. • Process messages with topic taxonomies • Easily handle multicast scenarios • Adjust characteristics separately on each topic Introduction to Messaging Topics Neuron ESB Inventory Topic Orders Topic Knowledge Base Topic
- 12. Introduction to Messaging Topics • Quality of service (Transport) • Durable Transports • MSMQ • RabbitMQ • Non-durable Transports • TCP • Named Pipes • Peer • Encryption • Auditing • Compression • Throttling • Data throughput • Messages per second Topics are hierarchical (i.e. Company.Finance.Invoices)
- 13. MSMQ • Supports transactions and durability • Support non-durable / in memory • Supports store and forward if Neuron ESB service is offline • Supports Pub/Sub even if Neuron ESB service is offline (Direct = true) • Supports automatic DLQ and Poison Message handling • Can be used for remotely hosted Neuron ESB Clients RabbitMQ • Supports transactions (publisher confirms, TX) and durability • Supports non-durable / in memory • Supports store and forward if Neuron ESB service is offline • Supports DLQ message handling • Supports mirrored nodes • Supports duplicate detection • Can be used for remotely hosted Neuron ESB Clients Introduction to Messaging Topic Transports : Durable
- 14. TCP • Allows for Server / Client port configuration • Supports WCF Ordered, Reliable Sessions, Encryption • Can be used for remotely hosted Neuron ESB Clients • Good for Request/Response Adapter & Service Endpoints Named Pipes • No need for port configuration • Can NOT be used for remotely hosted Neuron ESB Clients • Good for Request/Response Adapter & Service Endpoints Introduction to Messaging Topic Transports : Non-Durable Peer • WCF Peer Network Protocol (not UDP) • Neuron provides in Memory and Persisted Peer Resolver • Scales out to hundreds/thousands of subscribers
- 15. Single Instance • MSMQ • RabbitMQ Multiple Instances • TCP • Named Pipes • Peer Introduction to Messaging Topic Instances
- 16. Pull Subscription Model • MSMQ • RabbitMQ Push Subscription Model • TCP • Named Pipes • Peer Introduction to Messaging Topic Subscription Models
- 17. Introduction to Messaging When to use each topic transport TCP/IP • Where in memory routing is needed • Request/Response MEP • Port Configuration MSMQ • Where “Atomic” transactions are required • Need to inspect/edit messages waiting to be delivered RabbitMQ • Durability where “Atomic” transactions are not required • No desire to use Windows Clustering for HA • No need to inspect/edit messages waiting to be delivered Named Pipes • Where in memory routing is needed • Request/Response MEP • ONLY on local Neuron Server Peer • Broadcast MEP • In memory routing
- 18. Introduction to Messaging Parties • A party can be either a publisher, subscriber or both • Publishers send topical messages to the bus • Subscribers receive topical messages from the bus • Control access to topics • Determine which processes run on messages and when they run • Determine which conditions to use when filtering messages • A party is defined by what it can do with a message in the context of a given topic • For a given topic, a party can be both a publisher and subscriber
- 19. Configurable items for a party • Subscriptions • Security • Controls who is allowed to interact with that party • Business Processes • Allows logic to be applied to a message before it has been published to the bus or after it has been received from the bus Introduction to Messaging Parties
- 20. • Conditions provide the ability to add message filters to publishers and subscribers • Can be subscription specific (i.e. Ad Hoc) • Used for content based routing • Can be saved and reused across any subscription • Conditions are part of a Party’s Topic subscription • Message Properties • Message Body • Multiple filters for a condition • Multiple Predicates • (equals, greater than, xpath, etc..) Introduction to Messaging Conditions
- 21. • Subscriptions are added to parties through the Neuron ESB Explorer • Each selected topic is a subscription • Each subscription has a permission • Send • Receive • Send + Receive • Exclude • A subscription can have one or more reusable conditions or Ad Hoc conditions • Subscriptions support wildcard topics • Account.* • Account.*.Loans • Account.Company.Loans.* • Account.*.Loan.* Introduction to Messaging Subscriptions
- 22. Topics : Demo Purpose: Familiarize users with topics and configurations inside of the Neuron ESB Explorer. Objectives: Acquaint users with the following aspects of Topics • Hierarchy • Transports • Throttling • Auditing • Security • Dependencies
- 23. Introduction to Messaging Topic Taxonomy • Topic taxonomy is an important detail in any integration or ESB project. • It is highly recommended that a hierarchical topic taxonomy be used to control how messages are routed across the bus. • Benefits • It allows for more meaningful self-describing subscriptions which can model events, systems, categories or functions • Hierarchical topics can be easier to manage, providing the hierarchy doesn’t extend beyond 3 to 5 levels • They are more intuitive as they can be modeled to represent specific resources, people or systems to route messages to • Subtopics can be used to describe different versions of existing services that messages may be routed to • It reduces resource utilization on the Neuron ESB Service
- 24. Introduction to Messaging Topic Taxonomy Guidance When creating a Topic Taxonomy for your organization some key points to keep in mind are: • Orderly • Well documented • Contextual relationships • Can differ across message population • Name sub-topics to be reusable not specific
- 25. Introduction to Messaging Topic Taxonomy Business & Technical Requirements Determining the appropriate Topic Taxonomy usually requires that the business requirements of the application are taken into consideration of the development team. There are however technical requirements which will always govern the minimum number of Topics required to be developed for any application. They are: • QOS or Network Transport requirements: For instance, if there are some messages that require transactions and others that don’t, then a Root Topic must be developed for each. Also, Compression, Encryption and Throttling are all Topic level properties. Lastly, MSMQ based Topics SHOULD NOT be used for Request/Response style messaging. • Number of Subscribers.: Many of the Network Transports for Topics (Peer will decrease as well, but will tend to flatten out after 4 subscribers) will decrease in overall Throughput and increase latency as the number of subscribers increases. This is due to how the list of subscribers are resolved and dispatched to within the Neuron ESB runtime. • Message Throughput: Directly affected by the number subscribers. Increasing message Throughput may mean adding root topics, providing that the number of subscribers was indeed impacting Throughput.
- 26. End to End Messaging : Demo Purpose: Demonstrate end to end messaging inside the Neuron ESB explorer. Objectives: Acquaint users with following aspect of Messaging • Creating topics • Creating parties • Assigning subscriptions to parties • Creating and assigning conditions • Testing messages using the Neuron Test Client
- 27. Introduction to Messaging : Lab Purpose: Learn how to create topics, and sub-topics, assign subscriptions for both publishers and subscribers to a topic and how to use the Neuron Test Client to send messages between publishers and subscribers in order to test your topics and subscriptions. Objectives: • Create new topics • Create new publishers • Create new subscribers • Use the Neuron Test Client to test subscriptions
- 28. Introduction to Messaging Review • Messaging in Neuron ESB is a Pub / Sub based model • Messages are sent across topics with parties determining what endpoints have access to those topics via subscriptions. • Conditions can be used to determine whether or when to receive a message. • Neuron ESB supports both durable and non-durable message transports • Durable transports are single instance while non-durable transports support multiple instances of a subscriber. • Durable transports use a pull subscription model while non-durable transports use a push subscription model. • Topics should be well organized according to business logic