Salesforce Marketing Cloud - Why Einstein Engagement Scoring alone isn't enough

Potentials and challenges of AI-supported customer segmentation in Salesforce Marketing Cloud

Einstein Engagement Scoring (EES) is intended to provide valuable insights into the engagement of contacts and predictions about the intensity of contact interaction through AI-based analysis of customer interactions (cf. our blog post from 2020). The AI divides contacts into different categories: from particularly active recipients to occasional responders to less active contacts and addresses in need of reactivation. This should help marketers plan more targeted and effective campaigns.

Officially, email engagement, subscriber retention, web conversion, and overall engagement are included in the assessment. However, the actual weighting of these factors remains opaque and observations made by our customers make us doubt the usability of the AI categorizations: A comprehensive data analysis of the segmentations carried out by EES among our customers shows inconsistencies and a dominance of email metrics.

A remarkable number of contacts who are rated as “very active” have shown openings in the last 6 months, but hardly any click interactions or conversions. Focusing on the opening rate, whose importance has declined sharply in the context of e-privacy development, seems to us to be of little use.

For us, the central problem of Einstein Engagement Scoring lies in its black box character. As a user, you have no insight into the weighting of the individual factors and have only limited options for adjusting the scoring parameters.

Einstein Engagement Scoring: Limitations in Practical Application

Demand and reality also diverge when it comes to updating. The documentation promises weekly updates, while in practice we see different delays depending on the business unit, sometimes with significant latency times. In the case of web conversion, which, according to the documentation, should be fully integrated, the actual integration of this data remains unclear and difficult to understand.

Einstein Engagement Scoring also works in isolation within the respective business unit of a Salesforce Marketing Cloud instance. A comprehensive analysis of customer behavior across several business units is therefore not possible. This can lead to a distorted view of customer engagement when journeys and campaigns have been set up in different business units. The lack of transparency in the ML model also makes it difficult to trace and optimize segmentation. In addition, there are the minimum technical requirements for the amount of data (1,000 events in 90 days), which can be problematic for smaller campaigns or niche segments.

Users who use EES for campaign planning should set up detailed and continuous monitoring and keep a close eye on the performance of their journeys. Because inaccuracies in segmentation can, in the worst case, result in valuable contacts being excluded from communication.

We recommend that our customers not use EES to manage their communication but to set up a tailor-made scoring model for such far-reaching decisions that fits the respective setup, business model and target groups.

That suits you: Create scoring models yourself

The development of a tailor-made scoring model starts with the careful definition of the relevant KPIs, which we develop together with our customers. In contrast to the standardized ESS approach, company-specific metrics can be integrated in this way. By analyzing and incorporating CRM data, buying behavior and individual points of interaction, a custom scoring model enables a significantly more differentiated customer evaluation that is precisely tailored to a company's business model.

The technical planning phase involves developing a data model that is tailored to customer needs. The weighting of individual factors, the update frequency and the integration points into existing systems can be individually defined. This flexibility is a decisive advantage over the rigid structure of the ESS.

The technical implementation is typically carried out using SQL Automation Studio, which enables complex calculations. API integrations make it possible to integrate real-time data and, thanks to the flexible scoring logic and transparent weighting factors, you can understand at any time how your segmentation comes about — a clear contrast to the black box character of ESS.

Validation and continuous optimization

A major advantage of tailor-made models lies in the extensive testing and validation options. By building an A/B testing framework and setting up control groups, the effectiveness of the scoring model can be continuously checked. Performance monitoring and established feedback loops ensure that segmentations remain precise not only initially but also permanently.

With custom scoring models, the optimization phase is not the end, but a continuous process. You can adjust weightings, integrate new data sources, react to market changes and consider company-specific exceptions — all aspects that are not possible with EES or are only possible to a very limited extent.

Practical recommendations for action

If you want to work with Einstein Engagement Scoring despite the above restrictions, we recommend:

First, define clear goals and carry out a test phase with measurable KPIs. Regular validation of the generated segments and the combination with other data sources can increase the reliability of the results.

Careful resource planning is essential to develop your own scoring model. Define clear success criteria and rely on an agile development model that enables rapid adjustments. Continuous monitoring ensures that your model delivers precise results over the long term.

Conclusion: Realistic expectations instead of AI euphoria

Einstein Engagement Scoring should be used with realistic expectations. In practice, the AI revolution in customer segmentation promised by Salesforce turns out to be a limited tool with significant transparency weaknesses.

While Einstein Engagement Scoring may be suitable for simple email personalization based on email engagement, deeper analyses reveal significant limits. Although a proprietary scoring model requires higher initial investments, it enables better control, greater transparency, business-specific adjustments and more sustainable ROI development in the long term.

The decision between EES and custom scoring should be made based on your company-specific requirements, available resources, and long-term marketing strategy. However, our experience shows that investing in a tailor-made model pays off in the long term for most companies.