As a solutions architect, We often encounter situations where Salesforce is one of many authorities operating within an enterprise system. To ensure our clients can meet the promise of seamless customer experiences, my colleagues and we must integrate Salesforce with other corporate systems, eventually taking some responsibility but passing control to other platforms at various points during the customer journey. Under such conditions, we must guarantee system integration and smooth data transfers between platforms. Event-driven architectures in Salesforce Integrations allow you to quickly develop decoupled and robust APIs, making it simple for use cases to span across several systems within your company.
Salesforce indeed boasts an exceptional API, but as an architect, We prefer systems that feature loosely connected components or layers of abstraction between them. That can sometimes be problematic with architects but that subject lies outside our scope here. So feel free to Google it yourself for more info.
Change Data Capture And Platform Events For Salesforce Integrations
In this article, we'll describe how Platform Events and Change Data Capture (CDC), two powerful newcomers to Salesforce's arsenal, can help integrate it with various systems within an organization sporadically.
One client requested the creation of a 360-degree image that included various external and internal systems of their customer, such as:
- Data files are acquired from other sources (governmental bodies or third-party administrators).
- Master data management system (MDM)
- Training management and delivery system
- Data warehouse system
- Salesforce Marketing Cloud
Mastering customer data enabled seamless transfers between platforms, including Salesforce. This required frequent data exchanges between Salesforce and several platforms such as G Suite or Box; we will focus only on Salesforce's process components - mainly using Platform Events and CDC to accomplish that end since complete flow analysis or specific exchange scenarios would fall outside this discussion.
So, we will demonstrate how event-driven architecture can help create a loosely linked data exchange mechanism between Salesforce and another system - such as MDM systems - whereby both can exchange their mastered client record and all associated information like employment or credential data.
Architectural Approach
We came up with a few essential factors for the architectural strategy.
Publish / Subscribe Design Philosophy
Enterprise architects wanted a publish-subscribe design philosophy and loosely coupled systems as their goal, so using platform events on Salesforce to facilitate publishing from third-party systems allowed me to achieve it quickly as a Salesforce Solutions Architect. we could establish contact records within Salesforce that corresponded with client records quickly published from those systems.
A Salesforce consulting services provider can publish an event with the Contact record's record ID so that third-party systems can use it to complete a data exchange handshake. No system depends on another for completion and receives credit when its tasks are accomplished. No additional systems are needed.
Cost Considerations
Due to Salesforce costs, we couldn't simply create multiple event types and apply publishing/processing logic for each one - this would only increase costs further. Instead, we needed a creative solution; an exciting addition to the Platform Events framework is Change Data Capture (CDC), which makes publishing change events simple by activating it on items requiring it. Once activated on specific items within the Platform Events framework, it will handle everything else automatically.
Maintain a thorough knowledge of anticipated changes and daily event counts you post, recognizing when extra event blocks might become necessary in our instance, we performed cost-benefit analyses and volume projections to prove this justification of costs.
Processing Control
At first, our strategy focused on using both CDC and Platform Events; however, due to data exchange scenarios with MDM systems, we needed more control over when these events were published for specific objects. Salesforce consulting services provide this control by immediately publishing CDC events when end users make or modify actions in Salesforce, making things clear.
So as not to cause Salesforce another event, we needed to ensure MDM issued updates for already synced contacts; otherwise, an endless cycle would develop between both systems as they received update events one after the next, requiring us to adapt our strategy accordingly.
At last, we utilized Salesforce to subscribe to CDC events created by ourselves, extract the payload, wrap it with another event type (and only publish) when required, and then publish only as required through an MDM system that would subscribe directly to this wrapper event instead of subscribing directly.
Diagnostics, Tracking, and Error Handling
Error handling, tracking, and diagnostics complete the puzzle of effective data transference. By carefully scrutinizing event streams and events being sent both outwardly and inwardly to identify whether data was being transmitted correctly, we decided to create our logging system; something incredibly inventive yet simple like our logger saved hours and prevented blame games between two systems that often arise when trying to transmit their messages; for instance when sending or not sending correctly have you received it yet or is it sent correctly.
Design
After we established our strategy, it was necessary to craft an affordable architecture. After considering several approaches, we ultimately settled upon basing the JSON structure of CDC Events on Platform Events; that way, processing logic would remain relatively consistent regardless of event type, further driving down implementation costs through scale economies that resulted in reduced implementation costs; furthermore, standard design patterns for class hierarchy proved highly adaptable - we only required two essential platform events from MDM/Salesforce in terms of publishing/wrapper.
Platform Events Design
At the core of its design was an event featuring various generic features to connect two systems:
Header
- Entity the change pertained to (We had four entities.)
- Type of change (Create or Update for now; more options later.)
- Change source (MDM or Salesforce)
Payload
- Field Name/Field Value pairs of the changes for any given entity
- Each system would publish changes it deems appropriate
- The subscribing system would decide which changes it is interested in and how to process such changes
Incoming Events Processing
On the Salesforce side, we utilized an Apex class hierarchy for managing both inbound and outbound events. We are processing incoming events involving one derived class and one base class per entity.
Below are the duties assigned to the introductory class:
Orchestrating the common processing logic:
- Deserializing JSON payload
- Reading the header to determine entity type and change type
- Delegating to the appropriate derived entity class at specific points in the logic using virtual methods
Infrastructural work:
- Logging a record for each incoming event processed
- Publishing error events
Derived classes provided the entity-specific logic necessary for creating records of various kinds for specific entities, including procedures needed to create records from scratch or using data properties to associate existing parent records with an item being processed. Most of this labor was handled through overridden virtual methods that the base class called as part of processing logic logic.
Outgoing Events Processing
Processing outgoing events was more straightforward while using an identical design. Each entity contained one base class and two derived classes.
The following are the responsibilities of primary class students:
Coordinating The Logic Of Standard Processing
- Create a platform event by including the CDC event payload from Salesforce in an HTML tag.
- releasing the final event
- Utilizing virtual methods at crucial points in their reasoning to assign relevant derived classes at specific intervals is also feasible.
Work On Infrastructure
- keeping a record of every processed outgoing event
- releasing error events
Derived classes took care of any specific logic relating to an entity and its particular circumstances that must be managed, like when NOT to publish events based on these. For instance, do not publish wrapper events when this transaction involves contact update rather than contact creation; all such work was accomplished using overridden virtual methods that the base class accessed at various points during processing logic.
Logging Incoming/ Outgoing Events For Diagnostics
We designed a Log Entry object to store essential event details, like replay ID, event type, and payload size information, to make event tracking as effortless and long-term system maintainability as possible.
Logging events to this object was the starting point of event processing, providing us with an effective means of creating reports on this object and recording all incoming and outgoing events into and from our system while viewing its payload JSON using any JSON viewer to compare input with output.
As stated, this saved us many hours of debugging time, unneeded tension between teams, and heartburn from failing to observe events as they unfolded in real time. As an added benefit, we implemented a custom metadata item that allowed Salesforce specialists to pause or resume recording when desired, similar to how debug logs work. Limiting excessive logging for optimal efficiency when designing any system is best.
Error Events
Once we released custom error events that recorded relevant information at certain junctures in apex classes - such as error codes and descriptions - this proved future-proof: errors could now be handled similarly to regular data events by MDM systems; for instance, they might submit these events directly into shared error reporting infrastructures or correct mistakes before sending back into Salesforce once more as data events, among many other options available to them.
On the Salesforce side, as every event is logged to Log Entry, we could view an error event report and devise an approach to triaging and fixing different kinds of errors spanning both systems - informing MDM data stewards when needed.
Want More Information About Our Services? Talk to Our Consultants!
Conclusion
Event-driven architectures allow you to quickly develop decoupled and robust APIs, making it simple for use cases to span across several systems within your company. Contact us as we are the best Salesforce Consulting Services Company.
